{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "s_qNSzzyaCbD"
   },
   "source": [
    "##### Copyright 2019 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "colab": {},
    "colab_type": "code",
    "id": "jmjh290raIky"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "J0Qjg6vuaHNt"
   },
   "source": [
    "# Neural Machine Translation with Attention"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "AOpGoE2T-YXS"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://www.tensorflow.org/beta/tutorials/text/nmt_with_attention\">\n",
    "    <img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />\n",
    "    View on TensorFlow.org</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/r2/tutorials/text/nmt_with_attention.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />\n",
    "    Run in Google Colab</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/r2/tutorials/text/nmt_with_attention.ipynb\">\n",
    "    <img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />\n",
    "    View source on GitHub</a>\n",
    "  </td>\n",
    "  <td>\n",
    "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/r2/tutorials/text/nmt_with_attention.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CiwtNgENbx2g"
   },
   "source": [
    "This notebook trains a sequence to sequence (seq2seq) model for Spanish to English translation. This is an advanced example that assumes some knowledge of sequence to sequence models.\n",
    "\n",
    "After training the model in this notebook, you will be able to input a Spanish sentence, such as *\"¿todavia estan en casa?\"*, and return the English translation: *\"are you still at home?\"*\n",
    "\n",
    "The translation quality is reasonable for a toy example, but the generated attention plot is perhaps more interesting. This shows which parts of the input sentence has the model's attention while translating:\n",
    "\n",
    "<img src=\"https://tensorflow.org/images/spanish-english.png\" alt=\"spanish-english attention plot\">\n",
    "\n",
    "Note: This example takes approximately 10 mintues to run on a single P100 GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tnxXKDjq3jEL"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorflow-gpu==2.0.0-beta1 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (2.0.0b1)\n",
      "Requirement already satisfied: six>=1.10.0 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.12.0)\n",
      "Requirement already satisfied: tb-nightly<1.14.0a20190604,>=1.14.0a20190603 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.14.0a20190603)\n",
      "Requirement already satisfied: google-pasta>=0.1.6 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (0.1.7)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.1.0)\n",
      "Requirement already satisfied: wheel>=0.26 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (0.33.6)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.1.0)\n",
      "Requirement already satisfied: grpcio>=1.8.6 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.23.0)\n",
      "Requirement already satisfied: wrapt>=1.11.1 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.11.2)\n",
      "Requirement already satisfied: absl-py>=0.7.0 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (0.8.0)\n",
      "Requirement already satisfied: numpy<2.0,>=1.14.5 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.17.2)\n",
      "Requirement already satisfied: keras-applications>=1.0.6 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.0.8)\n",
      "Requirement already satisfied: tf-estimator-nightly<1.14.0.dev2019060502,>=1.14.0.dev2019060501 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (1.14.0.dev2019060501)\n",
      "Requirement already satisfied: gast>=0.2.0 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (0.2.2)\n",
      "Requirement already satisfied: astor>=0.6.0 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (0.8.0)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tensorflow-gpu==2.0.0-beta1) (3.9.2)\n",
      "Requirement already satisfied: markdown>=2.6.8 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow-gpu==2.0.0-beta1) (3.1.1)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow-gpu==2.0.0-beta1) (0.16.0)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from tb-nightly<1.14.0a20190604,>=1.14.0a20190603->tensorflow-gpu==2.0.0-beta1) (41.2.0)\n",
      "Requirement already satisfied: h5py in d:\\anaconda3\\envs\\py37-tf2\\lib\\site-packages (from keras-applications>=1.0.6->tensorflow-gpu==2.0.0-beta1) (2.10.0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "D:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "!pip install tensorflow-gpu==2.0.0-beta1\n",
    "import tensorflow as tf\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import unicodedata\n",
    "import re\n",
    "import numpy as np\n",
    "import os\n",
    "import io\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wfodePkj3jEa"
   },
   "source": [
    "## Download and prepare the dataset\n",
    "\n",
    "We'll use a language dataset provided by http://www.manythings.org/anki/. This dataset contains language translation pairs in the format:\n",
    "\n",
    "```\n",
    "May I borrow this book?\t¿Puedo tomar prestado este libro?\n",
    "```\n",
    "\n",
    "There are a variety of languages available, but we'll use the English-Spanish dataset. For convenience, we've hosted a copy of this dataset on Google Cloud, but you can also download your own copy. After downloading the dataset, here are the steps we'll take to prepare the data:\n",
    "\n",
    "1. Add a *start* and *end* token to each sentence.\n",
    "2. Clean the sentences by removing special characters.\n",
    "3. Create a word index and reverse word index (dictionaries mapping from word → id and id → word).\n",
    "4. Pad each sentence to a maximum length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kRVATYOgJs1b"
   },
   "outputs": [],
   "source": [
    "# Download the file\n",
    "path_to_zip = tf.keras.utils.get_file(\n",
    "    'spa-eng.zip', origin='http://storage.googleapis.com/download.tensorflow.org/data/spa-eng.zip',\n",
    "    extract=True)\n",
    "\n",
    "path_to_file = os.path.dirname(path_to_zip)+\"/spa-eng/spa.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rd0jw-eC3jEh"
   },
   "outputs": [],
   "source": [
    "# Converts the unicode file to ascii\n",
    "def unicode_to_ascii(s):\n",
    "    return ''.join(c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn')\n",
    "\n",
    "\n",
    "def preprocess_sentence(w):\n",
    "    w = unicode_to_ascii(w.lower().strip())\n",
    "\n",
    "    # creating a space between a word and the punctuation following it\n",
    "    # eg: \"he is a boy.\" => \"he is a boy .\"\n",
    "    # Reference:- https://stackoverflow.com/questions/3645931/python-padding-punctuation-with-white-spaces-keeping-punctuation\n",
    "    w = re.sub(r\"([?.!,¿])\", r\" \\1 \", w)\n",
    "    w = re.sub(r'[\" \"]+', \" \", w)\n",
    "\n",
    "    # replacing everything with space except (a-z, A-Z, \".\", \"?\", \"!\", \",\")\n",
    "    w = re.sub(r\"[^a-zA-Z?.!,¿]+\", \" \", w)\n",
    "\n",
    "    w = w.rstrip().strip()\n",
    "\n",
    "    # adding a start and an end token to the sentence\n",
    "    # so that the model know when to start and stop predicting.\n",
    "    w = '<start> ' + w + ' <end>'\n",
    "    return w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "opI2GzOt479E"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<start> may i borrow this book ? <end>\n",
      "b'<start> \\xc2\\xbf puedo tomar prestado este libro ? <end>'\n"
     ]
    }
   ],
   "source": [
    "en_sentence = u\"May I borrow this book?\"\n",
    "sp_sentence = u\"¿Puedo tomar prestado este libro?\"\n",
    "print(preprocess_sentence(en_sentence))\n",
    "print(preprocess_sentence(sp_sentence).encode('utf-8'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OHn4Dct23jEm"
   },
   "outputs": [],
   "source": [
    "# 1. Remove the accents\n",
    "# 2. Clean the sentences\n",
    "# 3. Return word pairs in the format: [ENGLISH, SPANISH]\n",
    "def create_dataset(path, num_examples):\n",
    "    lines = io.open(path, encoding='UTF-8').read().strip().split('\\n')\n",
    "\n",
    "    word_pairs = [[preprocess_sentence(w) for w in l.split('\\t')]  for l in lines[:num_examples]]\n",
    "\n",
    "    return zip(*word_pairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cTbSbBz55QtF"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<start> if you want to sound like a native speaker , you must be willing to practice saying the same sentence over and over in the same way that banjo players practice the same phrase over and over until they can play it correctly and at the desired tempo . <end>\n",
      "<start> si quieres sonar como un hablante nativo , debes estar dispuesto a practicar diciendo la misma frase una y otra vez de la misma manera en que un musico de banjo practica el mismo fraseo una y otra vez hasta que lo puedan tocar correctamente y en el tiempo esperado . <end>\n"
     ]
    }
   ],
   "source": [
    "en, sp = create_dataset(path_to_file, None)\n",
    "print(en[-1])\n",
    "print(sp[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OmMZQpdO60dt"
   },
   "outputs": [],
   "source": [
    "def max_length(tensor):\n",
    "    return max(len(t) for t in tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bIOn8RCNDJXG"
   },
   "outputs": [],
   "source": [
    "def tokenize(lang):\n",
    "  lang_tokenizer = tf.keras.preprocessing.text.Tokenizer(\n",
    "      filters='')\n",
    "  lang_tokenizer.fit_on_texts(lang)\n",
    "\n",
    "  tensor = lang_tokenizer.texts_to_sequences(lang)\n",
    "\n",
    "  tensor = tf.keras.preprocessing.sequence.pad_sequences(tensor,\n",
    "                                                         padding='post')\n",
    "\n",
    "  return tensor, lang_tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eAY9k49G3jE_"
   },
   "outputs": [],
   "source": [
    "def load_dataset(path, num_examples=None):\n",
    "    # creating cleaned input, output pairs\n",
    "    targ_lang, inp_lang = create_dataset(path, num_examples)\n",
    "\n",
    "    input_tensor, inp_lang_tokenizer = tokenize(inp_lang)\n",
    "    target_tensor, targ_lang_tokenizer = tokenize(targ_lang)\n",
    "\n",
    "    return input_tensor, target_tensor, inp_lang_tokenizer, targ_lang_tokenizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GOi42V79Ydlr"
   },
   "source": [
    "### Limit the size of the dataset to experiment faster (optional)\n",
    "\n",
    "Training on the complete dataset of >100,000 sentences will take a long time. To train faster, we can limit the size of the dataset to 30,000 sentences (of course, translation quality degrades with less data):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cnxC7q-j3jFD"
   },
   "outputs": [],
   "source": [
    "# Try experimenting with the size of that dataset\n",
    "num_examples = 30000\n",
    "input_tensor, target_tensor, inp_lang, targ_lang = load_dataset(path_to_file, num_examples)\n",
    "\n",
    "# Calculate max_length of the target tensors\n",
    "max_length_targ, max_length_inp = max_length(target_tensor), max_length(input_tensor)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4QILQkOs3jFG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24000 24000 6000 6000\n"
     ]
    }
   ],
   "source": [
    "# Creating training and validation sets using an 80-20 split\n",
    "input_tensor_train, input_tensor_val, target_tensor_train, target_tensor_val = train_test_split(input_tensor, target_tensor, test_size=0.2)\n",
    "\n",
    "# Show length\n",
    "print(len(input_tensor_train), len(target_tensor_train), len(input_tensor_val), len(target_tensor_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lJPmLZGMeD5q"
   },
   "outputs": [],
   "source": [
    "def convert(lang, tensor):\n",
    "  for t in tensor:\n",
    "    if t!=0:\n",
    "      print (\"%d ----> %s\" % (t, lang.index_word[t]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VXukARTDd7MT"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input Language; index to word mapping\n",
      "1 ----> <start>\n",
      "16 ----> esta\n",
      "9061 ----> empezando\n",
      "10 ----> a\n",
      "761 ----> llover\n",
      "3 ----> .\n",
      "2 ----> <end>\n",
      "\n",
      "Target Language; index to word mapping\n",
      "1 ----> <start>\n",
      "10 ----> it\n",
      "11 ----> s\n",
      "2360 ----> starting\n",
      "15 ----> to\n",
      "353 ----> rain\n",
      "3 ----> .\n",
      "2 ----> <end>\n"
     ]
    }
   ],
   "source": [
    "print (\"Input Language; index to word mapping\")\n",
    "convert(inp_lang, input_tensor_train[0])\n",
    "print ()\n",
    "print (\"Target Language; index to word mapping\")\n",
    "convert(targ_lang, target_tensor_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rgCLkfv5uO3d"
   },
   "source": [
    "### Create a tf.data dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TqHsArVZ3jFS"
   },
   "outputs": [],
   "source": [
    "BUFFER_SIZE = len(input_tensor_train)\n",
    "BATCH_SIZE = 64\n",
    "steps_per_epoch = len(input_tensor_train)//BATCH_SIZE\n",
    "embedding_dim = 256\n",
    "units = 1024\n",
    "vocab_inp_size = len(inp_lang.word_index)+1\n",
    "vocab_tar_size = len(targ_lang.word_index)+1\n",
    "\n",
    "dataset = tf.data.Dataset.from_tensor_slices((input_tensor_train, target_tensor_train)).shuffle(BUFFER_SIZE)\n",
    "dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qc6-NK1GtWQt"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(TensorShape([64, 16]), TensorShape([64, 11]))"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "example_input_batch, example_target_batch = next(iter(dataset))\n",
    "example_input_batch.shape, example_target_batch.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TNfHIF71ulLu"
   },
   "source": [
    "## Write the encoder and decoder model\n",
    "\n",
    "Implement an encoder-decoder model with attention which you can read about in the TensorFlow [Neural Machine Translation (seq2seq) tutorial](https://github.com/tensorflow/nmt). This example uses a more recent set of APIs. This notebook implements the [attention equations](https://github.com/tensorflow/nmt#background-on-the-attention-mechanism) from the seq2seq tutorial. The following diagram shows that each input words is assigned a weight by the attention mechanism which is then used by the decoder to predict the next word in the sentence. The below picture and formulas are an example of attention mechanism from [Luong's paper](https://arxiv.org/abs/1508.04025v5). \n",
    "\n",
    "<img src=\"https://www.tensorflow.org/images/seq2seq/attention_mechanism.jpg\" width=\"500\" alt=\"attention mechanism\">\n",
    "\n",
    "The input is put through an encoder model which gives us the encoder output of shape *(batch_size, max_length, hidden_size)* and the encoder hidden state of shape *(batch_size, hidden_size)*.\n",
    "\n",
    "Here are the equations that are implemented:\n",
    "\n",
    "<img src=\"https://www.tensorflow.org/images/seq2seq/attention_equation_0.jpg\" alt=\"attention equation 0\" width=\"800\">\n",
    "<img src=\"https://www.tensorflow.org/images/seq2seq/attention_equation_1.jpg\" alt=\"attention equation 1\" width=\"800\">\n",
    "\n",
    "This tutorial uses [Bahdanau attention](https://arxiv.org/pdf/1409.0473.pdf) for the encoder. Let's decide on notation before writing the simplified form:\n",
    "\n",
    "* FC = Fully connected (dense) layer\n",
    "* EO = Encoder output\n",
    "* H = hidden state\n",
    "* X = input to the decoder\n",
    "\n",
    "And the pseudo-code:\n",
    "\n",
    "* `score = FC(tanh(FC(EO) + FC(H)))`\n",
    "* `attention weights = softmax(score, axis = 1)`. Softmax by default is applied on the last axis but here we want to apply it on the *1st axis*, since the shape of score is *(batch_size, max_length, hidden_size)*. `Max_length` is the length of our input. Since we are trying to assign a weight to each input, softmax should be applied on that axis.\n",
    "* `context vector = sum(attention weights * EO, axis = 1)`. Same reason as above for choosing axis as 1.\n",
    "* `embedding output` = The input to the decoder X is passed through an embedding layer.\n",
    "* `merged vector = concat(embedding output, context vector)`\n",
    "* This merged vector is then given to the GRU\n",
    "\n",
    "The shapes of all the vectors at each step have been specified in the comments in the code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nZ2rI24i3jFg"
   },
   "outputs": [],
   "source": [
    "class Encoder(tf.keras.Model):\n",
    "  def __init__(self, vocab_size, embedding_dim, enc_units, batch_sz):\n",
    "    super(Encoder, self).__init__()\n",
    "    self.batch_sz = batch_sz\n",
    "    self.enc_units = enc_units\n",
    "    self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
    "    self.gru = tf.keras.layers.GRU(self.enc_units,\n",
    "                                   return_sequences=True,\n",
    "                                   return_state=True,\n",
    "                                   recurrent_initializer='glorot_uniform')\n",
    "\n",
    "  def call(self, x, hidden):\n",
    "    x = self.embedding(x)\n",
    "    output, state = self.gru(x, initial_state = hidden)\n",
    "    return output, state\n",
    "\n",
    "  def initialize_hidden_state(self):\n",
    "    return tf.zeros((self.batch_sz, self.enc_units))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "60gSVh05Jl6l"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Encoder output shape: (batch size, sequence length, units) (64, 16, 1024)\n",
      "Encoder Hidden state shape: (batch size, units) (64, 1024)\n"
     ]
    }
   ],
   "source": [
    "encoder = Encoder(vocab_inp_size, embedding_dim, units, BATCH_SIZE)\n",
    "\n",
    "# sample input\n",
    "sample_hidden = encoder.initialize_hidden_state()\n",
    "sample_output, sample_hidden = encoder(example_input_batch, sample_hidden)\n",
    "print ('Encoder output shape: (batch size, sequence length, units) {}'.format(sample_output.shape))\n",
    "print ('Encoder Hidden state shape: (batch size, units) {}'.format(sample_hidden.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "umohpBN2OM94"
   },
   "outputs": [],
   "source": [
    "class BahdanauAttention(tf.keras.Model):\n",
    "  def __init__(self, units):\n",
    "    super(BahdanauAttention, self).__init__()\n",
    "    self.W1 = tf.keras.layers.Dense(units)\n",
    "    self.W2 = tf.keras.layers.Dense(units)\n",
    "    self.V = tf.keras.layers.Dense(1)\n",
    "\n",
    "  def call(self, query, values):\n",
    "    # hidden shape == (batch_size, hidden size)\n",
    "    # hidden_with_time_axis shape == (batch_size, 1, hidden size)\n",
    "    # we are doing this to perform addition to calculate the score\n",
    "    hidden_with_time_axis = tf.expand_dims(query, 1)\n",
    "\n",
    "    # score shape == (batch_size, max_length, 1)\n",
    "    # we get 1 at the last axis because we are applying score to self.V\n",
    "    # the shape of the tensor before applying self.V is (batch_size, max_length, units)\n",
    "    score = self.V(tf.nn.tanh(\n",
    "        self.W1(values) + self.W2(hidden_with_time_axis)))\n",
    "\n",
    "    # attention_weights shape == (batch_size, max_length, 1)\n",
    "    attention_weights = tf.nn.softmax(score, axis=1)\n",
    "\n",
    "    # context_vector shape after sum == (batch_size, hidden_size)\n",
    "    context_vector = attention_weights * values\n",
    "    context_vector = tf.reduce_sum(context_vector, axis=1)\n",
    "\n",
    "    return context_vector, attention_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "k534zTHiDjQU"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attention result shape: (batch size, units) (64, 1024)\n",
      "Attention weights shape: (batch_size, sequence_length, 1) (64, 16, 1)\n"
     ]
    }
   ],
   "source": [
    "attention_layer = BahdanauAttention(10)\n",
    "attention_result, attention_weights = attention_layer(sample_hidden, sample_output)\n",
    "\n",
    "print(\"Attention result shape: (batch size, units) {}\".format(attention_result.shape))\n",
    "print(\"Attention weights shape: (batch_size, sequence_length, 1) {}\".format(attention_weights.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yJ_B3mhW3jFk"
   },
   "outputs": [],
   "source": [
    "class Decoder(tf.keras.Model):\n",
    "  def __init__(self, vocab_size, embedding_dim, dec_units, batch_sz):\n",
    "    super(Decoder, self).__init__()\n",
    "    self.batch_sz = batch_sz\n",
    "    self.dec_units = dec_units\n",
    "    self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
    "    self.gru = tf.keras.layers.GRU(self.dec_units,\n",
    "                                   return_sequences=True,\n",
    "                                   return_state=True,\n",
    "                                   recurrent_initializer='glorot_uniform')\n",
    "    self.fc = tf.keras.layers.Dense(vocab_size)\n",
    "\n",
    "    # used for attention\n",
    "    self.attention = BahdanauAttention(self.dec_units)\n",
    "\n",
    "  def call(self, x, hidden, enc_output):\n",
    "    # enc_output shape == (batch_size, max_length, hidden_size)\n",
    "    context_vector, attention_weights = self.attention(hidden, enc_output)\n",
    "\n",
    "    # x shape after passing through embedding == (batch_size, 1, embedding_dim)\n",
    "    x = self.embedding(x)\n",
    "\n",
    "    # x shape after concatenation == (batch_size, 1, embedding_dim + hidden_size)\n",
    "    x = tf.concat([tf.expand_dims(context_vector, 1), x], axis=-1)\n",
    "\n",
    "    # passing the concatenated vector to the GRU\n",
    "    output, state = self.gru(x)\n",
    "\n",
    "    # output shape == (batch_size * 1, hidden_size)\n",
    "    output = tf.reshape(output, (-1, output.shape[2]))\n",
    "\n",
    "    # output shape == (batch_size, vocab)\n",
    "    x = self.fc(output)\n",
    "\n",
    "    return x, state, attention_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "P5UY8wko3jFp"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Decoder output shape: (batch_size, vocab size) (64, 4935)\n"
     ]
    }
   ],
   "source": [
    "decoder = Decoder(vocab_tar_size, embedding_dim, units, BATCH_SIZE)\n",
    "\n",
    "sample_decoder_output, _, _ = decoder(tf.random.uniform((64, 1)),\n",
    "                                      sample_hidden, sample_output)\n",
    "\n",
    "print ('Decoder output shape: (batch_size, vocab size) {}'.format(sample_decoder_output.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_ch_71VbIRfK"
   },
   "source": [
    "## Define the optimizer and the loss function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WmTHr5iV3jFr"
   },
   "outputs": [],
   "source": [
    "optimizer = tf.keras.optimizers.Adam()\n",
    "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(\n",
    "    from_logits=True, reduction='none')\n",
    "\n",
    "def loss_function(real, pred):\n",
    "  mask = tf.math.logical_not(tf.math.equal(real, 0))\n",
    "  loss_ = loss_object(real, pred)\n",
    "\n",
    "  mask = tf.cast(mask, dtype=loss_.dtype)\n",
    "  loss_ *= mask\n",
    "\n",
    "  return tf.reduce_mean(loss_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DMVWzzsfNl4e"
   },
   "source": [
    "## Checkpoints (Object-based saving)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Zj8bXQTgNwrF"
   },
   "outputs": [],
   "source": [
    "checkpoint_dir = './training_checkpoints'\n",
    "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt\")\n",
    "checkpoint = tf.train.Checkpoint(optimizer=optimizer,\n",
    "                                 encoder=encoder,\n",
    "                                 decoder=decoder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hpObfY22IddU"
   },
   "source": [
    "## Training\n",
    "\n",
    "1. Pass the *input* through the *encoder* which return *encoder output* and the *encoder hidden state*.\n",
    "2. The encoder output, encoder hidden state and the decoder input (which is the *start token*) is passed to the decoder.\n",
    "3. The decoder returns the *predictions* and the *decoder hidden state*.\n",
    "4. The decoder hidden state is then passed back into the model and the predictions are used to calculate the loss.\n",
    "5. Use *teacher forcing* to decide the next input to the decoder.\n",
    "6. *Teacher forcing* is the technique where the *target word* is passed as the *next input* to the decoder.\n",
    "7. The final step is to calculate the gradients and apply it to the optimizer and backpropagate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sC9ArXSsVfqn"
   },
   "outputs": [],
   "source": [
    "@tf.function\n",
    "def train_step(inp, targ, enc_hidden):\n",
    "  loss = 0\n",
    "\n",
    "  with tf.GradientTape() as tape:\n",
    "    enc_output, enc_hidden = encoder(inp, enc_hidden)\n",
    "\n",
    "    dec_hidden = enc_hidden\n",
    "\n",
    "    dec_input = tf.expand_dims([targ_lang.word_index['<start>']] * BATCH_SIZE, 1)\n",
    "\n",
    "    # Teacher forcing - feeding the target as the next input\n",
    "    for t in range(1, targ.shape[1]):\n",
    "      # passing enc_output to the decoder\n",
    "      predictions, dec_hidden, _ = decoder(dec_input, dec_hidden, enc_output)\n",
    "\n",
    "      loss += loss_function(targ[:, t], predictions)\n",
    "\n",
    "      # using teacher forcing\n",
    "      dec_input = tf.expand_dims(targ[:, t], 1)\n",
    "\n",
    "  batch_loss = (loss / int(targ.shape[1]))\n",
    "\n",
    "  variables = encoder.trainable_variables + decoder.trainable_variables\n",
    "\n",
    "  gradients = tape.gradient(loss, variables)\n",
    "\n",
    "  optimizer.apply_gradients(zip(gradients, variables))\n",
    "\n",
    "  return batch_loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ddefjBMa3jF0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "Epoch 1 Batch 0 Loss 0.0879\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "Epoch 1 Batch 100 Loss 0.0421\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "Epoch 1 Batch 200 Loss 0.0606\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "Epoch 1 Batch 300 Loss 0.0392\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "Epoch 1 Loss 0.0778\n",
      "Time taken for 1 epoch 61.16939043998718 sec\n",
      "\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n",
      "Epoch 2 Batch 0 Loss 0.0753\n",
      "train_step (64, 16) (64, 11) (64, 1024)\n",
      "train_step <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'> <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-35-481314102fc7>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m     10\u001b[0m     \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'train_step'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarg\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0menc_hidden\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     11\u001b[0m     \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'train_step'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0minp\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtarg\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0menc_hidden\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 12\u001b[1;33m     \u001b[0mbatch_loss\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mtrain_step\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0minp\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtarg\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0menc_hidden\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     13\u001b[0m     \u001b[0mtotal_loss\u001b[0m \u001b[1;33m+=\u001b[0m \u001b[0mbatch_loss\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     14\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\eager\\def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, *args, **kwds)\u001b[0m\n\u001b[0;32m    402\u001b[0m       \u001b[1;31m# In this case we have created variables on the first call, so we run the\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    403\u001b[0m       \u001b[1;31m# defunned version which is guaranteed to never create variables.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 404\u001b[1;33m       \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_stateless_fn\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m*\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[1;33m)\u001b[0m  \u001b[1;31m# pylint: disable=not-callable\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    405\u001b[0m     \u001b[1;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_stateful_fn\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    406\u001b[0m       \u001b[1;31m# In this case we have not created variables on the first call. So we can\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36m__call__\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m   1333\u001b[0m     \u001b[1;34m\"\"\"Calls a graph function specialized to the inputs.\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1334\u001b[0m     \u001b[0mgraph_function\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_maybe_define_function\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1335\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_filtered_call\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0margs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m  \u001b[1;31m# pylint: disable=protected-access\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1336\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1337\u001b[0m   \u001b[1;33m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36m_filtered_call\u001b[1;34m(self, args, kwargs)\u001b[0m\n\u001b[0;32m    587\u001b[0m     \"\"\"\n\u001b[0;32m    588\u001b[0m     return self._call_flat(\n\u001b[1;32m--> 589\u001b[1;33m         (t for t in nest.flatten((args, kwargs), expand_composites=True)\n\u001b[0m\u001b[0;32m    590\u001b[0m          if isinstance(t, (ops.Tensor,\n\u001b[0;32m    591\u001b[0m                            resource_variable_ops.ResourceVariable))))\n",
      "\u001b[1;32mD:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[1;34m(self, args)\u001b[0m\n\u001b[0;32m    669\u001b[0m     \u001b[1;31m# Only need to override the gradient in graph mode and when we have outputs.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    670\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mcontext\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mexecuting_eagerly\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mor\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0moutputs\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 671\u001b[1;33m       \u001b[0moutputs\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_inference_function\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcall\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mctx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0margs\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    672\u001b[0m     \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    673\u001b[0m       \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_register_gradient\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\eager\\function.py\u001b[0m in \u001b[0;36mcall\u001b[1;34m(self, ctx, args)\u001b[0m\n\u001b[0;32m    443\u001b[0m             attrs=(\"executor_type\", executor_type,\n\u001b[0;32m    444\u001b[0m                    \"config_proto\", config),\n\u001b[1;32m--> 445\u001b[1;33m             ctx=ctx)\n\u001b[0m\u001b[0;32m    446\u001b[0m       \u001b[1;31m# Replace empty list with None\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    447\u001b[0m       \u001b[0moutputs\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0moutputs\u001b[0m \u001b[1;32mor\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mD:\\Anaconda3\\envs\\py37-tf2\\lib\\site-packages\\tensorflow\\python\\eager\\execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[1;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[0;32m     59\u001b[0m     tensors = pywrap_tensorflow.TFE_Py_Execute(ctx._handle, device_name,\n\u001b[0;32m     60\u001b[0m                                                \u001b[0mop_name\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mattrs\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 61\u001b[1;33m                                                num_outputs)\n\u001b[0m\u001b[0;32m     62\u001b[0m   \u001b[1;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[1;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m     63\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "EPOCHS = 10\n",
    "\n",
    "for epoch in range(EPOCHS):\n",
    "  start = time.time()\n",
    "\n",
    "  enc_hidden = encoder.initialize_hidden_state()\n",
    "  total_loss = 0\n",
    "\n",
    "  for (batch, (inp, targ)) in enumerate(dataset.take(steps_per_epoch)):\n",
    "#     print('train_step', inp.shape, targ.shape, enc_hidden.shape)\n",
    "#     print('train_step', type(inp), type(targ), type(enc_hidden))\n",
    "    batch_loss = train_step(inp, targ, enc_hidden)\n",
    "    total_loss += batch_loss\n",
    "\n",
    "    if batch % 100 == 0:\n",
    "        print('Epoch {} Batch {} Loss {:.4f}'.format(epoch + 1,\n",
    "                                                     batch,\n",
    "                                                     batch_loss.numpy()))\n",
    "  # saving (checkpoint) the model every 2 epochs\n",
    "  if (epoch + 1) % 2 == 0:\n",
    "    checkpoint.save(file_prefix = checkpoint_prefix)\n",
    "\n",
    "  print('Epoch {} Loss {:.4f}'.format(epoch + 1,\n",
    "                                      total_loss / steps_per_epoch))\n",
    "  print('Time taken for 1 epoch {} sec\\n'.format(time.time() - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mU3Ce8M6I3rz"
   },
   "source": [
    "## Translate\n",
    "\n",
    "* The evaluate function is similar to the training loop, except we don't use *teacher forcing* here. The input to the decoder at each time step is its previous predictions along with the hidden state and the encoder output.\n",
    "* Stop predicting when the model predicts the *end token*.\n",
    "* And store the *attention weights for every time step*.\n",
    "\n",
    "Note: The encoder output is calculated only once for one input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "EbQpyYs13jF_"
   },
   "outputs": [],
   "source": [
    "def evaluate(sentence):\n",
    "    attention_plot = np.zeros((max_length_targ, max_length_inp))\n",
    "\n",
    "    sentence = preprocess_sentence(sentence)\n",
    "\n",
    "    inputs = [inp_lang.word_index[i] for i in sentence.split(' ')]\n",
    "    inputs = tf.keras.preprocessing.sequence.pad_sequences([inputs],\n",
    "                                                           maxlen=max_length_inp,\n",
    "                                                           padding='post')\n",
    "    inputs = tf.convert_to_tensor(inputs)\n",
    "\n",
    "    result = ''\n",
    "\n",
    "    hidden = [tf.zeros((1, units))]\n",
    "    enc_out, enc_hidden = encoder(inputs, hidden)\n",
    "\n",
    "    dec_hidden = enc_hidden\n",
    "    dec_input = tf.expand_dims([targ_lang.word_index['<start>']], 0)\n",
    "\n",
    "    for t in range(max_length_targ):\n",
    "        predictions, dec_hidden, attention_weights = decoder(dec_input,\n",
    "                                                             dec_hidden,\n",
    "                                                             enc_out)\n",
    "\n",
    "        # storing the attention weights to plot later on\n",
    "        attention_weights = tf.reshape(attention_weights, (-1, ))\n",
    "        attention_plot[t] = attention_weights.numpy()\n",
    "\n",
    "        predicted_id = tf.argmax(predictions[0]).numpy()\n",
    "\n",
    "        result += targ_lang.index_word[predicted_id] + ' '\n",
    "\n",
    "        if targ_lang.index_word[predicted_id] == '<end>':\n",
    "            return result, sentence, attention_plot\n",
    "\n",
    "        # the predicted ID is fed back into the model\n",
    "        dec_input = tf.expand_dims([predicted_id], 0)\n",
    "\n",
    "    return result, sentence, attention_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "s5hQWlbN3jGF"
   },
   "outputs": [],
   "source": [
    "# function for plotting the attention weights\n",
    "def plot_attention(attention, sentence, predicted_sentence):\n",
    "    fig = plt.figure(figsize=(10,10))\n",
    "    ax = fig.add_subplot(1, 1, 1)\n",
    "    ax.matshow(attention, cmap='viridis')\n",
    "\n",
    "    fontdict = {'fontsize': 14}\n",
    "\n",
    "    ax.set_xticklabels([''] + sentence, fontdict=fontdict, rotation=90)\n",
    "    ax.set_yticklabels([''] + predicted_sentence, fontdict=fontdict)\n",
    "\n",
    "    ax.xaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "    ax.yaxis.set_major_locator(ticker.MultipleLocator(1))\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sl9zUHzg3jGI"
   },
   "outputs": [],
   "source": [
    "def translate(sentence):\n",
    "    result, sentence, attention_plot = evaluate(sentence)\n",
    "\n",
    "    print('Input: %s' % (sentence))\n",
    "    print('Predicted translation: {}'.format(result))\n",
    "\n",
    "    attention_plot = attention_plot[:len(result.split(' ')), :len(sentence.split(' '))]\n",
    "    plot_attention(attention_plot, sentence.split(' '), result.split(' '))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n250XbnjOaqP"
   },
   "source": [
    "## Restore the latest checkpoint and test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UJpT9D5_OgP6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.training.tracking.util.CheckpointLoadStatus at 0x274a93dc288>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# restoring the latest checkpoint in checkpoint_dir\n",
    "checkpoint.restore(tf.train.latest_checkpoint(checkpoint_dir))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WrAM0FDomq3E"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> hace mucho frio aqui . <end>\n",
      "Predicted translation: it s very cold here . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAJwCAYAAAC08grWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZilB1nn/d8dOgmGRQZQiAuCAoZFlqQFWUZgcMQBV143DAriS1BhBMUNGcfIvAFBUHHQkaDCsDksA4Ogg+xGBcSAssgSYiABEZJo1ISQkOV+/3hOm6qiOiShU/fprs/nuvq6qp5z6tRdTzp9vvWs1d0BAJhw2PQAAMDuJUQAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCZA1U1W2q6k1V9XXTswDAThIi6+FhSe6b5BHDcwDAjio3vZtVVZXko0len+TbknxZd182OhQA7BBbRObdL8kNkvxEkkuTPHB2HADYOUJk3g8leXl3X5jkD7LspgGAXcGumUFVdb0k/5DkQd39Z1V1lyRvy7J75rzZ6QDg2meLyKz/J8m53f1nSdLdf5Pkw0m+f3QqAA56VXW9qvqhqvri6VmujBCZ9YNJXrhl2Qtj9wwAX7jvTfLcLO81a8uumSFV9ZVJPpLkdt394Q3LvyLLWTS37+7ThsZjDVTVnZL8dJLbJ+kk70/y9O5+7+hgwEGhqt6S5EuTXNjde4fH2S8hAmuoqr49ySuS/FmSP18tvvfqz4O7+9VTswHrr6pumeS0JHdL8vYkx3b3+ydn2h8hMqiqbpHkY73Nf4SqukV3nzUwFmugqt6T5JXd/Utblj8pyXd0951nJgMOBlX1i0nu2933r6pXJPlwd//c9FzbcYzIrI8k+ZKtC6vqJqvH2L1um+QF2yx/QZKv3eFZgIPPD+WKf0NemOT41QU0144QmVVZ9v1vdf0kF+3wLKyXs5Mct83y45J8aodnAQ4iVXXPJEcnedlq0WuSHJXkm8aGuhJ7pgfYjarqN1cfdpKnVNWFGx6+TpZ9en+z44OxTp6T5NlVdeskb83yd+XeWQ5e/dXJwYC197Akr+ruTydJd3+2ql6a5OFZbieyVhwjMqCq3rz68D5ZLmD22Q0PfzbLWTNP33g2DbvLahPq45I8PsmXrRZ/IkuE/OZ2xxUBVNWRST6Z5CHd/doNy++d5E+S3Ky7L5iabztCZMjqjealSR7R3edPz8P6qqobJIm/J8DnU1U3zXLPshd29+VbHntokjd09ydHhtsPITKkqq6T5TiQO6/rKVUAcG1zjMiQ7r6sqs5McsT0LKyfqrpxkpOS3D/LBYk2HVje3TecmAvgQBMis/5bkl+pqod297nTw7BWfi/JXZOcnOXYEJsugf2qqo/kKv470d1ffS2Pc7XYNTOoqt6b5FZJDk/y8SSf3vh4d99pYi7mVdW/JvmP3f2X07MA66+qHr/h0+sn+akk78hyQkSS3CPLGZnP6O4n7fB4V8oWkVkvnx6AtXV2krU6sh1YX939jH0fV9Xzkjy1u5+88TlV9YQkd9jh0T4vW0RgDVXV92W5c+bD1u1UO2C9rbaoHtvdp29Zfusk71q3Y8xsEWFtVNWPJ3l0lt1Vd+zuM6rq55Oc0d0vnZ3u2rfaVbfxN4NbJTl7dVDzJRufa7cdcCU+neS+SU7fsvy+SS7c+uRpQmRQVR2R5IlJHpLkFlmOFfk33X2dibkmVNXjkvxskqcm+ZUND/19ksdkuebKoc6uOuBA+PUkv1VVe7PceTdJviHLFVdPnBpqf+yaGVRVT03yfUmekuUvzn9Jcssk35/kF7v72XPT7ayq+mCSx3f3H1XV+Vmur3JGVd0hySndfZPhEWFUVR2b5G+6+/LVx/vV3e/aobFYU1X1vUkem+R2q0UfSPLMddy6LEQGrU63+rHufu3qzfcu3f13VfVjSe7f3d89POKOqarPJDmmu8/cEiK3zfKP71HDI+6oqrpPknT3n26zvLv7lJHBGFNVlye5eXefvfq4s9w4c6veTVtTOfjZNTPrZkn2XVX1giQ3Wn382iy7KHaTM5Icm+TMLcsfmCvW0W7y60m2O8Xuhlk2rW53Z14ObbdKcs6Gj+Hzqqob5XMviPhPQ+NsS4jMOivLDc3OynJQ0QOSvDPL+d6fGZxrwtOTPKuqjsryW949quoHsxw38ojRyWZ8bZJ3b7P8vavH2GW6+8ztPoatquqrkvxOkvtl87GHlWVL2lptMRMis16Z5RLeb0/yzCR/UFWPTPLl2WW3eu/u51bVniRPTnJUkhdkOVD1J7r7JaPDzfhMlkj9yJblX5HNd2tmF3KMCJ/Hc7NsYX9EDoIrMztGZI1U1d2T3CvJad39mul5pqzuHnlYd589PcuUqnpRljOpvr27z1stu3GS/5Pk77v7IZPzMWs/x4j82z/mjhHZ3arqgiTf0N3vm57lqhAig6rqG5O8tbsv3bJ8T5J77qYDEldnx1ynu9+zZfmdkly62+5QXFVHJzklyw3v9q2TO2W54up9uvsTU7Mxb7XpfaPDs9yb6IlJntDd/3fnp2JdrK5J9PDufuf0LFeFEBlUVZclOXrrb/5VdZMkZ++m32qq6i+S/FZ3v3jL8u9P8pjuvvfMZHNWx8scn+QuWX7zfVeSF3f32l2QaCdU1X9Icvssv/m/v7vfPDzS2qmqb07yS919r+lZmLP6f+Xnk/z41qurriMhMmi1efVm3X3OluW3TXLqul2G99q0OmX3rttckvhrslyS+ItnJmNaVX15luOpjsuyvztZjp85Ncl32Tp0haq6TZbT3a83PQtzVv+eHpnloNSLk2za6r5u7y0OVh1QVX+4+rCTvLCqLt7w8HWS3DHJW3d8sFmXJdkuNv5dtr9WwiGtqh58ZY939yt2apY18JtZ/n7curs/kiRV9dVJXrh6bNdcb2ef1fFCmxYlOTrLqd0f2vGBWDePmR7g6rBFZEBVPXf14cOyXLp846m6n03y0STP6e5zd3i0MVX1qixvNt/T3Zetlu1J8rIkh3f3t07Ot9NWW8u208nuOhhxdQOv+249E2R1+eo37satZRsOVt20OMnHknxfd7/9c78K1pMtIgO6+4eTpKo+muTp3f3p2YnWws8m+fMkp1fVn6+W3TvJ9ZN849hUQ7p70wWIVlF21yyndT9xZKj1s79Y2w3ut+Xzy7Nc7Oz0rQe/sztV1c2S/GCSr8lyy5Bzq+peST6xb8viurBFZFBVHZYk3X356vObJ/nWLAfi7bZdM/vOFHlMNh+c+duOAbhCVd0zyf/o7jtPz7JTquqVSb4kyUO6+2OrZbdI8qIk53T3le7Ggt2mqo5L8sYs1yG6Q5bbZ5xRVScmuW13/8DkfFsJkUFV9X+TvLa7n1lV10/ywSTXy7IV4Ee6+/mjA7J2qur2Sd7R3defnmWnVNVXJnlVkq/LFRdn+vIspzV/R3d/fHC8EatT/6+S3XQZABZV9eYsNwv9pS337rpHkv/V3VtP/x5l18ys47LskkiSByf51yz3kDg+yU8n2XUhUlVfluVCXkdsXL7b/jHd5sqZ+w5G/Lkkf73zE81ZbQU5tqr+Y5JjsqyL93f3G2YnG/WWXHGMyL6Dubd+vm/ZrjmeiH9zXJIf2Wb5P2S5x9laESKzbpDkn1cff3OSV3b3JVX1piS/NTfWzlsFyIuzHA+y74qRGzfX7bZ/TE/N9ndXfXt257130t2vT/L66TnWxLdmuT/TSUnetlp2jyS/kOWXGwer7m6fyXLG4VbHZLko4loRIrPOSnKvqnp1lhvefc9q+Y2T7LaLVv1GlrNmbp/kr5J8S5Zyf1KSnxyca8rWu6tenuV4iIsmhtlpVfVTWY4Pumj18X5196/t0Fjr5L8leewqzvY5o6rOTvK07r7r0Fysh1cl+aWq2vee0lV1yyx3df/fU0Ptj2NEBlXVo5I8K8kFSc5Mcmx3X15VP5HkO7v7P4wOuIOq6lNJHtTdp65O19zb3adV1YOyHPH9DcMj7rjVwcv3zHKZ96238f7tkaF2SFV9JMvfgX9cfbw/3d1fvVNzrYuq+kyWfy8+sGX57ZO8s7u/aGYy1kFV3TDJH2e5LcT1knwyyy92b03yn9btTE0hMmx1dPMtkry+uy9YLXtQkn/u7r8YHW4HreLjTt390dVpzQ/t7j+vqlsl+dvuPmp2wp1VVQ9N8rtZds2cl827qbq7v2xkMNZCVZ2a5PQkP9zdn1kt+6Isd129dXfvnZyP9bC61PuxWX6Rede6Hldl18yQqvriLG+8f5Zk642J/jnJrrrJW5Yzho7JcjG3v0nyo1X1sSSPTvL3g3NNOSnJ05I8aTdfF6KqDs9yfZkf6m5XDL3CjyV5TZK/r6p9N0X8uiy7Nx80NhXjNr63dPebkrxpw2P3ynKg93ljA27DFpEhVXWDLEcwP2Djlo+qukuSv0zy5bvsyqrHZ7mC6vNWZ4y8NslNs9wn4WHd/dLRAXdYVZ2X5LjuPmN6lmmr4x7u3d2nTc+yTjbcFPF2WZ1JlOWmiGu12Z2ddTC+twiRQVX1oiQXdPejNix7epYLznz73GTzVv/IHpPkrHX7n2YnVNWzknyou//79CzTqupXk6S7f2Z6lnWyutru3bL96e677tR/rnCwvbcIkUFV9YAkf5DlDryXrK60+vEst73fTTc1S5JU1fcluX+2Pzhz7f7nuTZV1RFJ/k+Wew+9N8klGx/v7idNzDWhqn47y2/+H8myG3PTb/zd/RMTc02qqmOSvDrL2VWVZZfMnix/Ty5et7ursrMOtvcWx4jMen2W03S/LckrsrwJH5HlH5hdZfVb7+OSvDlXXD1zN3tUllOYz01y62w5WDXLac2HrNWVQ9+6Oj7mdlku958kW8+Q2a1/T34jS5TdJcsZEXfJcvfq/5HkvwzOxXo4qN5bbBEZVlVPTfK13f2dVfX8JOd396On59ppq9N3H93dL5+eZR2sjot4Snf/+vQsE6rqsiRHd/fZVXVGkq/v7n+cnmtdVNU/JrlPd7+vqv4lyd26+0NVdZ8k/7277zQ8IsMOpvcWW0TmPT/JO1f30/iuLOW6Gx2W5WwZFtdJ8ofTQww6L8tuh7OT3DJbdtWRyhUXPTwny713PpRl8/utp4ZirRw07y22iKyBqvqrJBcluWl33256nglVdVKSS7r7xOlZ1sHqwLJ/3U3HgmxUVc9O8rAsR//fIssb7GXbPXeXXtDslCS/3t2vrKoXJ7lJkicneWSWUzdtEeGgeW+xRWQ9vCDLPt8nTg+yk6rqNzd8eliS41c3NntPPvfgzN12QOJRSf7f1UFnu3F9/GiWLUK3SfJrWS7Udf7oROvlpCxXzEyWY0Jek+X4qnOTfO/UUOumqj6Q5DbdvVvf6w6K95bd+h9n3bwwyw2Knjs9yA77ui2f79s1c8yW5btxs93tcsVddnfd+uhlU+0fJUlV3TnJM7pbiKx0959s+PiMJLevqhsnOa9t5t7ot7JsLdqtDor3FrtmAIAxDgADAMYIEQBgjBBZE1V1wvQM68T62Mz62Mz62Mz62Mz62Gzd14cQWR9r/RdlgPWxmfWxmfWxmfWxmfWx2VqvDyECAIzZ9WfNHFFH9nX/7XT8OZfk4hyeI6fHWBvWx2bWx2bWx2bWx2bWx2brsj7Oz3nndveXbF2+668jct1cL3evtb3yLbDOqqYnWC+7/Bdbrtwb+uVnbrfcrhkAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYIwQAQDGCBEAYMwhESJV9byqes30HADA1bNneoAD5LFJKkmq6i1J3tfdjxmdCAD4vA6JEOnuf5meAQC4+g6JEKmq5yW5aZJzk9wnyX2q6tGrh2/V3R8dGg0AuBKHRIhs8Ngkt03ywSS/sFp2ztw4AMCVOaRCpLv/pao+m+TC7v7k/p5XVSckOSFJrpujdmo8AGCLQ+Ksmauru0/u7r3dvffwHDk9DgDsWrsyRACA9XAohshnk1xneggA4PM7FEPko0nuVlW3rKqbVtWh+DMCwCHhUHyTfnqWrSLvz3LGzC1mxwEA9ueQOGumux++4ePTktxjbhoA4Ko6FLeIAAAHCSECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIwRIgDAGCECAIzZMz3AtDriiOz5iq+aHmNtfPZ3L58eYa1c+rSbTY+wVr7ob86aHmGt9IWfmR5hrVz+6QunR1gvl182PcFBwRYRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxhxyIVJV31hVb6+qC6rqX6rqL6vqjtNzAQCfa8/0AAdSVe1J8qokv5fk+CSHJzk2yWWTcwEA2zukQiTJDZPcKMmru/vvVss+uPVJVXVCkhOS5Lp7brBz0wEAmxxSu2a6+5+SPC/Jn1TVH1XVT1XVV27zvJO7e2937z3isKN2fE4AYHFIhUiSdPcPJ7l7klOSfHuS06rqAbNTAQDbOeRCJEm6+93d/dTuvm+StyR52OxEAMB2DqkQqapbVdWvVNU9q+qrqup+Se6U5P3TswEAn+tQO1j1wiS3TfKyJDdN8qkkL0ry1MmhAIDtHVIh0t2fSvLg6TkAgKvmkNo1AwAcXIQIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBmz/QA4y69NJd/6pzpKdbG4T929PQIa+V6z/nY9Ahr5Zzf+urpEdbKDV7119MjrJe+fHoCDkK2iAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAYw76EKmqI6ZnAACumR0Nkap6VFV9qqr2bFn+4qp61erjb6uqd1bVRVX1kao6aWNsVNVHq+rEqvr9qvrnJC+qqjdV1bO2vOYNq+rCqnrwjvxwAMDVttNbRF6a5EZJvmnfgqq6XpLvSPLCqnpAkhcleVaSOyR5RJLvTvLkLa/zU0k+mGRvkl9I8pwkP1BVR254zkOSXJDk1dfKTwIAfMF2NES6+7wkf5zk+A2LvyvJpVmC4YlJfrW7n9vdf9fdb07yc0l+tKpqw9f8aXc/rbtP7+4PJ3lFkstXr7XPI5I8v7sv2TpHVZ1QVadW1amf7YsO6M8IAFx1E8eIvDDJd1bVUavPj0/y8u6+KMlxSZ5YVRfs+5PkxUmul+TmG17j1I0v2N0XJ3lBlvhIVd0+yd2S/P52A3T3yd29t7v3HlHXPYA/GgBwdez5/E854F6TZQvId1TVG7Pspvnm1WOHJfnlJC/b5uvO2fDxp7d5/HeTvKeqbpHkR5K8rbvff8CmBgAOuB0Pke6+uKpenmVLyE2TfDLJn64efleSY7r79Gvwun9bVX+Z5JFJHpplNw8AsMYmtogky+6ZNyS5VZIXd/flq+VPSvKaqjozy4Gtlya5Y5K7dffPXoXXfU6S30lySZKXHPCpAYADauo6Iqck+fskt88SJUmS7v6TJA9Kcr8k71j9+fkkZ13F131Jks8meWl3n38gBwYADryRLSLd3UluuZ/HXpfkdVfytdt+3cqNknxRkt/7AsYDAHbI1K6ZA6qqDk9ydJKTkvx1d//F8EgAwFVw0F/ifeVeSc5McvcsB6sCAAeBQ2KLSHe/JUl9vucBAOvlUNkiAgAchIQIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY/ZMDzCtL788l1944fQY6+O0v5ueYK1c8rBbTI+wVp77p8+YHmGtPO5vHzE9wlq57AOnT4+wXvqy6QkOCraIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjhAgAMEaIAABjDsoQqaoTq+p9n+c5z6qqt+zQSADANXBQhggAcGgQIgDAmLEQqcXjq+rDVXVxVX28qp6yeuzrquoNVfWZqvqnqnpeVX3xlbzWdarq6VV13urPbyS5zo79MADANTK5ReTJSX4xyVOS3CHJ9yT5WFUdleS1SS5Icrck35Xknkl+/0pe6/FJHpnkUUnukSVCjr/WJgcADog9E9+0qq6f5CeTPK679wXG6UneVlWPTHL9JD/Y3eevnn9CkjdX1a27+/RtXvJxSZ7W3S9dPf+xSR5wJd//hCQnJMl1c9QB+qkAgKtraovI7ZMcmeSN2zx2uyTv2RchK29Ncvnq6zZZ7bI5Osnb9i3r7suT/OX+vnl3n9zde7t77+E58pr9BADAF2wqROrzPNb7eWx/ywGAg9BUiLw/ycVJ7r+fx+5cVTfYsOyeWWb9wNYnd/e/JPmHJN+wb1lVVZbjSwCANTZyjEh3n19Vz0zylKq6OMkpSW6S5Lgk/zPJLyd5flX91yT/Lsmzk7xiP8eHJMkzkzyhqk5L8t4kP55ld80/XLs/CQDwhRgJkZUnJDkvy5kzX5HkU0me390XVtUDkvxGknckuSjJq5I89kpe6xlJbp7kd1efvyDJi7IcbwIArKmxEFkdUPorqz9bH3tvtt9ts+/xE5OcuOHzS7OchfOTB3pOAODa48qqAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjNkzPQCss0s/etb0CGvlcXu/c3qEtfLH737J9Ahr5UF3/9bpEdbKZWefMz3Cerlo+8W2iAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY4QIADBGiAAAY3YsRKrqLVX1rJ36fgDA+rNFBAAYc1CHSFUdPj0DAHDN7XSIHFZVT66qc6vq7Kp6elUdliRVdURVPbWqPl5Vn66qv6qqB+z7wqq6b1V1VT2wqt5RVZ9N8oDVY99WVe+sqouq6iNVdVJVHbHDPxsAcDXt2eHvd3ySZya5Z5K7JHlxkncm+YMkz03yNUl+IMnHkzwwyaur6uu7+90bXuOpSR6f5PQk569i5UVJHpvklCS3SPI7SY5M8tPbDVFVJyQ5IUmum6MO7E8IAFxlOx0i7+/u/7r6+LSqemSS+1fVO5I8JMktu/us1ePPqqpvSvKoJD++4TVO7O7X7fukqp6Y5Fe7+7mrRX9XVT+X5IVV9TPd3VuH6O6Tk5ycJDesG3/O4wDAztjpEHnPls8/keRLkxybpJK8v6o2Pn5kkjdt+ZpTt3x+XJK7reJjn8OSfFGSmyf5hy9wZgDgWrLTIXLJls87SzQctvr467d5zme2fP7pLZ8fluSXk7xsm+93zjUbEwDYCTsdIvvz11m2iNy8u998Nb/2XUmO6e7TD/xYAMC1aS1CpLtPq6oXJXleVT0+S1zcOMl9k5zR3a+4ki9/UpLXVNWZSV6a5NIkd0xyt+7+2Wt3cgDgC7FO1xH54SxnzjwtyQeTvCbJNyY588q+qLv/JMmDktwvyTtWf34+yVlX9nUAwLwd2yLS3ffdZtnDN3x8SZITV3+2+/q3ZNl9s91jr0vyuu0eAwDW1zptEQEAdhkhAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCMESIAwBghAgCM2TM9AHDwuOycc6ZHWCvf8qDjp0dYK//5zf97eoS18qwHf+f0COvl3dsvtkUEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABgjRACAMUIEABizZ3qACVV1QpITkuS6OWp4GgDYvXblFpHuPrm793b33sNz5PQ4ALBr7coQAQDWgxABAMYIEQBgzCEbIlX1mKr64PQcAMD+HbIhkuSmSb52eggAYP8O2RDp7hO7u6bnAAD275ANEQBg/QkRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxl3i+A0AAAZ9SURBVAgRAGCMEAEAxggRAGCMEAEAxggRAGCMEAEAxggRAGDMnukBAA5W/e4PTo+wVo45/NzpEdbKRUdff3qE9fLu7RfbIgIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjBEiAMAYIQIAjDloQqSqfrqqPjo9BwBw4Bw0IQIAHHoOSIhU1Q2r6kYH4rWuxvf8kqq67k5+TwDgwLrGIVJV16mqB1TVi5N8MsmdV8u/uKpOrqqzq+r8qvrTqtq74eseXlUXVNX9q+p9VfXpqnpzVd1qy+v/bFV9cvXc5ye5/pYRHpjkk6vvda9r+nMAAHOudohU1R2q6mlJzkrykiSfTvItSU6pqkryR0m+PMm3JrlrklOSvKmqjt7wMkcmeUKSRyS5R5IbJfmdDd/je5P8f0l+KcmxST6U5Ke2jPKiJD+Q5AZJXl9Vp1fVf90aNPv5GU6oqlOr6tRLcvHVXQUAwAFylUKkqm5SVT9RVacm+eskxyR5XJKbdfcju/uU7u4k90tylyTf3d3v6O7Tu/sXk5yR5Ac3vOSeJI9ePec9SZ6e5H5VtW+exyX5n9397O4+rbtPSvKOjTN196Xd/cfd/ZAkN0vy5NX3//BqK8wjqmrrVpR9X3tyd+/t7r2H58irsgoAgGvBVd0i8p+TPDPJxUlu093f3t0v6+6tmxOOS3JUknNWu1QuqKoLktwxyddseN7F3f2hDZ9/IsnhWbaMJMntkrxty2tv/fzfdPf53f373X2/JF+f5EuT/F6S776KPx8AMGDPVXzeyUkuSfJDSf62ql6Z5AVJ3tjdl2143mFJPpXk32/zGv+64eNLtzzWG77+aquqI5M8KMtWlwcm+dssW1VedU1eDwDYGVfpjb+7P9HdJ3X31yb5piQXJPlfST5eVc+oqruunvquLLtJLl/tltn45+yrMdcHknzDlmWbPq/Fvavq2VkOln1WktOTHNfdx3b3M7v7vKvxPQGAHXa1t0B099u7+8eSHJ1ll81tk7yjqv59kjck+Yskr6qq/1RVt6qqe1TVL68ev6qemeRhVfXIqrpNVT0hyd23POehSV6X5IZJHpLkK7v7Z7r7fVf3ZwIAZlzVXTOfY3V8yMuTvLyqvjTJZd3dVfXALGe8PCfLsRqfyhInz78ar/2SqvrqJCdlOebkD5P8WpKHb3jaG5PcvLv/9XNfAQA4GFzjENlo426X7j4/yWNXf7Z77vOSPG/LsrckqS3LnpLkKVu+/MQNj3/imk8MAKwDl3gHAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgzJ7pAQAOWpdfNj3BWvnxr7r39Ahr5YicOj3CQcEWEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgjBABAMYIEQBgzJ7pASZU1QlJTkiS6+ao4WkAYPfalVtEuvvk7t7b3XsPz5HT4wDArrUrQwQAWA9CBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYI0QAgDFCBAAYU909PcOoqjonyZnTcyS5aZJzp4dYI9bHZtbHZtbHZtbHZtbHZuuyPr6qu79k68JdHyLroqpO7e6903OsC+tjM+tjM+tjM+tjM+tjs3VfH3bNAABjhAgAMEaIrI+TpwdYM9bHZtbHZtbHZtbHZtbHZmu9PhwjAgCMsUUEABgjRACAMUIEABgjRACAMUIEABjz/wMI41hDg/5qUwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'hace mucho frio aqui.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zSx2iM36EZQZ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> esta es mi vida . <end>\n",
      "Predicted translation: this is my life . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAJwCAYAAAAjo60MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debSlB1nn+9+TgcQQAheZAjLZQKOMHUoGsTGISxSV1XLRbiUQhktcXm2xafU2qxeKtEiDURsb2yaozNpgbtsog14QEFqmFRFBggaEMAgBogwJwSQkz/1j75LDSVWsc1JV77NPPp+1zqp93r3Prue8q6rOt96xujsAACzvmKUHAABgRZgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAaqqjtX1Ruq6h5LzwIAHD3CbKYzk5ye5PELzwEAHEXlJuazVFUluTDJ65J8b5Jbd/dViw4FABwVtpjN8+AkN0ry40m+nORhy44DABwtwmyexyQ5t7svS/I7We3WBACuB+zKHKSqbpjkk0m+u7vfUlX3TvK2rHZnfnbZ6QCAI80Ws1n+zyQXd/dbkqS7353kA0n+zaJTAcAGqaobVtVjqurGS8+yU8Jslkcneem2ZS+N3ZkAsBM/kOQFWf1c3Sh2ZQ5RVbdN8uEk39DdH9iy/OuyOkvzG7v7goXGA4CNUVVvSnKLJJd1976Fx9kRYQYA7BlVdYckFyS5b5K3Jzmtu89fcqadsCtzkKq63fo6Zgd87mjPAwAb6NFJ3rI+Tvs12bDDgYTZLB9OcvPtC6vqa9fPAQDX7jFJXrJ+/NIkjzrYRo+JhNksleRA+5ZPTvIPR3kWANgoVfXNSU5N8rvrRa9KclKSb19sqB06bukBSKrqV9cPO8kzq+qyLU8fm9V+8ncf9cEAYLOcmeSV3f3FJOnuK6rqFUkem9WtDscTZjPcY/1rJfmGJFdsee6KJO9KcvbRHgoANkVVnZDVZTJ+cNtTL03yR1V1cndfevQn2xlnZQ6x3v/9iiSP7+5Llp4HADZJVd0sq/tLv7S7r9723BlJXt/dFy0y3A4IsyGq6tisjiO71yad1gsAHD4O/h+iu69K8pEkN1h6FgBgGbaYDVJVZ2a1b/yM7r546XkAYLqq+nAOfEWDa+jurz/C41xnDv6f5SeT3DHJ31bVx5N8ceuT3X3PRaYCgLmeu+XxyUmenOSdSd62XvaArK5u8EtHea5dEWaznLv0AACwSbr7H4Orql6Y5Fnd/QtbX1NVT0lyt6M82q7YlQkA7AlV9YWs7o35wW3L75TkXd19yjKTHToH/wMAe8UXk5x+gOWnJ7nsAMvHsStzkKq6QZL/mNUJALdLcvzW57v72CXmAoAN8StJfq2q9iV5+3rZ/bO6I8DTlhpqJ4TZLP8pyb9O8sys/nD9VJI7JPk3SZ663FgAMF93P7uqLkzypKzuApAk709yZne/YrHBdsAxZoOsT/n9ke7+w6q6JMm9u/tvqupHkjykux+58IgjVdXj8pWtjF91HbhNODUa9rqqummS78yB/44+fZGhYChbzGa5ZZL9V/2/NMlN1o//MMmzFplouKr6qSRPSfK8JA9K8t+S3Gn92P1FYWFVdf8kr05yeZKbJ/nbJKeuP78wiTDjiKiqm2TbsfTd/fcLjXPIHPw/y0eT3Hr9+INJHrp+/IAkX1pkovmemOSs7n5KkiuTPLe7H57V9Wpuv+hkQJL8YpKXJblNVred+7astpydF//h5DCrqttX1Wur6h+S/F2Sz6w/Ll7/Op4tZrP8XpKHZHXA4nOS/E5VPTGrf9B+ccnBBvu6rC4kmKzidf+p0L+zXv7EJYYC/tE9kzyhu7uqrkpyQnd/qKr+nyS/nVW0weHygqz2Nj0+ySdyiHcEmESYDbLe6rP/8blV9bEkD0xyQXe/arnJRrsoyc2y2tr4kay2Lr47q92ZG/cXEvagK7Y8/lRWW7Lfn9XhGrc+4FfA7t03yf27+y+XHmS3hNkgVfWgJG/t7i8nSXe/I8k7quq4qnpQd7952QlHekOShyd5V5LfTPIrVfUDSU5LshFn4MAe964k35TkgiRvSvLzVXXLJGckec+Cc7E3fTjJCUsPcV04K3OQ9Wb+U7v709uWf22ST7uO2TVV1TFJjtkfs1X1r7Peypjked195ZLzwfXd+npSN+ruN1bVzZO8OF/5O/q47n7vogOyp1TVtyX5D0n+7+1X/98UwmyQqro6yS27+zPblt8lyXmbcCuJo62qbpfkY73tD3JVVZLbdvdHl5kMgKNtfampE5Icm9WZv1/e+vwm/By1K3OAqvr99cNO8tKqunzL08cmuXuStx71wTbDh7M69f7T25bfdP2crYwA1x8/tvQA15Uwm+Hv1r9Wks/mqy+NcUWS/53k+Ud7qA1ROfBB/idndWo+cJStL5Z9SLtjXASaw6m7X7T0DNeVMBugux+XJOvbSJzd3V9cdqL5qupX1w87yTOrauvNaY/N6sycdx/1wYAkee6WxycneXJWl69523rZA7L6O/pLR3kurgfWJ5c8Osk/S/LU7r64qh6Y5BPd/eFlp/unOcZskPWB7Onuq9ef3yrJ9yQ5v7vtytyiqt64fvitWf1jv/WU/CuyuqL42d39gaM8GrBFVb0wq0v+/MK25U9JcrfuPmORwdiTquo+Sf44q0NZ7pbkruvr5j0tyV26+4eWnO9QCLNBquq1Sf6wu59TVScn+askN8zqf5xP6O4XLzrgQFX1giRP6u4vLD0LcE1V9YUkp20/Q66q7pTkXZtwMDabY/2f9jd398+uTwS41zrMHpDkf3T3+DvC2JU5y32S/PT68SOSfCHJHZM8KslPZnWaOVvs3w28X1V9TVan4n+guz+yzFSbx3o7uKp6RJI/6O4r148Pqrv/51Eaa5N8McnpWd1mbqvTk1y2/cVwHd0nyRMOsPyTWd2PejxhNsuNknxu/fg7kvze+ofBG5L82nJjzbXeTfLO7v5vVXWDrI5juVuSK6rq+7r7tYsOOJT1tiPnJrlVVmf+nnstr+s4C/hAfiXJr62vZ/b29bL7JzkzydOWGoo960tJ/o8DLL9rrnn2/khuYj7LR5M8sKpumNUNzF+3Xn7T+J/lwTw0X/nH/uFZxe2tsvoH/2nLjLQRrLdD1N3H7L/o8/rxwT5E2QF097OzOhD7Hkl+ef1xjyRndrebmHO4vTLJz1bV/qv/d1XdIcmzkvy/Sw21E44xG6Sqfjirs5kuzeq+j6d199VV9eNJ/lV3f9uiAw5UVf+Q5E7d/fGq+o0kn+/uf7/+i/je7r7RogMOZb3t3vqknG9Ocot89X9uu7t/fZmpgCSpqlOSvCbJPbM6RvuirHZhvjXJd23CVQ/syhyku59XVecluV2S1+0/OzPJ3yR56nKTjXZRkrtX1Sez2gp01nr5yUncjungrLddqKozkvxGvnLNwa3/s+0kwgwWtD4R7FvWt2Y6Lav/PL2ru1+/7GSHTpgNUVU3TnLP7n5Lkj/b9vTnkpx/9KfaCL+V5OVJPpHkqqxOk06S+2V1VisHZr3tzjOSPDvJ0/ffn5VrWp+J+fXr60ddkmu52KyzMjlctv4c7e43JHnDlucemNWlpz672ICHSJjNcXWS11bVQ7v7T/cvrKp7Z/WH6zaLTTZYdz+9qv4yye2TvKK791/P7MtZHVPAAVhvu3ZKkheKsn/Sv01yyfrxxt8ih42xJ36OOvh/iO6+JKuDFh+z7akzkvxRd1989KfaGF9K8u1JXldVt10vu0FWx+pxcNbbzr0syXcvPcR03f2i7t5/z99/lVWk/c56+Vd9LDgme8xe+TkqzGZ5cZLvr6rjk3+8E8APJXnhkkNNVlWPSvKKJBdkdc2349dPHZOvXBOObay3XXtyku+qqv9VVf+pqn5m68fSww31paz+bftUVT2/qh609EDsaRv/c1SYzfK6rC6L8b3rzx+S1RaMP1hsovl+OskTu/vfZbUbbr+3J7n3MiNtBOttd344yXdmdVbm9yX5/i0fj1xwrrHWt8C5RVa7N2+T5PVV9ZGqemZV3W3Z6diDNv7nqDAbZH0W5svylc2wj07y8u52ltzB3TlfuTHyVpdmdTwQB2a97c5Tk/z77r5Fd9+9u++x5eOeSw83VXdf1t0v7e6HZRVnv5jVD86/WHYy9pq98HPUwf/zvDjJn62P+fm+rGqfg/tEkrtkdd23rR6U1WVGODDrbXeOTfL7Sw+xqarqxCTfltUlWu6S5GPLTsQetdE/R20xG6a735fkvUl+O8nHu/udC4803TlJfnV9KnSS3LaqzszqkgauKXVw1tvuvCCre9dyiKrqmKr6jqp6UZJPZfXn65NJvr2777jsdOxFm/5z1BazmV6S5L8k+Y9LDzJddz97fe2a1yU5Mckbk1ye5Ozudn/Rg7Dedu2kJP9XVT00yXuy7WK83f3ji0w12yeS3DjJa5M8LsmrtlyehV2oqvcnuXN3+xl+cBv7c9QtmQaqqptmdaDs87r7oqXn2QRVdVKSb8xqK/D53e2SD4fAetuZqnrjtTzdbpt2TVV1VlbXyvvc0rPsFVX1Y0m+trt/bulZptrkn6PCDABgCMeYAQAMIcwAAIYQZoOtj81gh6y3nbPOdsd62x3rbeess93ZxPUmzGbbuD9QQ1hvO2ed7Y71tjvW285ZZ7uzcetNmAEADHG9PyvzBnVCn5gbLj3GAV2Zy3N8Tlh6jI1jve2cdbY71tvuWG87Z53tzuT1dkk+e3F333z78uv9xelOzA1zv9qouzUAABvu9X3u9lviJbErEwBgDGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBAjw6yqTq+qrqqbXZfXAABskhFhVlVvqqrn7vDL3prk1CR/dwRGAgA46o5beoDd6u4rkly09BwAAIfL4lvMquqFSb41yY+ud012kjusn75XVb2jqi6rqvOq6rQtX/dVuzKr6sZV9ZKq+nRV/UNVfaiqfuJofz8AALu1eJgleVKStyV5QVa7Jk9N8rH1c89M8h+SnJbVLsuXVVUd5H1+Psk9knxPkrsmeXySvz1yYwMAHF6L78rs7s9X1RVJLuvui5Kkqu66fvqp3f3G9bKnJ/nfSW6T5OMHeKvbJ/nz7n7n+vMLD/Z7VtVZSc5KkhNz0uH4NgAArrMJW8yuzXu2PP7E+tdbHOS1v57kB6rqL6rq7Kr61oO9aXef0937unvf8TnhcM0KAHCdTA+zK7c87vWvB5y5u1+b1Vazs5PcLMmrq+oFR3Y8AIDDZ0qYXZHk2Ov6Jt19cXe/pLsfm+QJSc6sKpvEAICNsPgxZmsXJrlvVd0hyaXZRTCuj0F7V5L3ZfV9PSLJh7r78sM2JQDAETRli9nZWW01Oz/JZ5LcbhfvcXmSZyT5iyR/muRGSb73cA0IAHCkVXf/06/aw06pm/b96iFLjwEAXI+8vs/9s+7et335lC1mAADXe8IMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDHLf0AEurqhxz4olLj7FxvvDwey89wsY5+Yc/vvQIG+mY77546RE20tWXX770CJune+kJwBYzAIAphBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhNjrMquqFVfWqpecAADgcjlt6gOvoSUlq6SEAAA6HjQ6z7v780jMAABwue2ZXZlU9qKreXlWXVtXnq+odVXX3pWcEADhUG73FbL+qOi7JK5P8ZpJHJTk+yWlJrlpyLgCAndgTYZbklCQ3SfIH3f0362V/dbAXV9VZSc5KkhPrhkd+OgCAQ7DRuzL36+6/T/LCJH9UVa+uqidX1W2v5fXndPe+7t53g5xw1OYEALg2eyLMkqS7H5fkfknenOThSS6oqocuOxUAwKHbM2GWJN39F939rO4+Pcmbkpy57EQAAIduT4RZVd2xqv5zVX1zVd2+qh6c5J5Jzl96NgCAQ7VXDv6/LMldkvxukpsl+VSSlyV51pJDAQDsxEaHWXc/dsunj1hqDgCAw2FP7MoEANgLhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIY4bukBFnfssakbn7L0FBvnRr/3rqVH2DwfuPPSE2ykj/32jZceYSN93TOWnmDz1Ps/vPQIG+nqyy5beoQ9xRYzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGGJcmFXVm6rq16vql6rq76vqM1X1pKo6oap+rao+V1UfrapHr1//hqp67rb3OKWqLquqRyzzXQAA7Ny4MFt7VJJLktwvyX9O8l+S/K8kFyTZl+RFSX6jqm6d5PlJfqiqTtjy9T+Y5NIkf3A0hwYAuC6mhtn7uvtp3f2BJL+c5OIkV3b3c7r7g0menqSSfHOS/5nk6iTft+XrH5/kxd195YHevKrOqqrzquq8K67+0hH9RgAADtXUMHvP/gfd3Uk+neS9W5ZdmeSzSW7R3ZcneUlWMZaq+sYk903yWwd78+4+p7v3dfe+GxzzNUfmOwAA2KHjlh7gILZv6eqDLNsflr+R5D1VdbskT0jytu4+/8iOCABweE3dYrYj3f2+JO9I8sQkZ+RatpYBAEw1dYvZbjw/yX/PasvayxeeBQBgx/bEFrO1lye5IskruvuSpYcBANipcVvMuvv0Ayy7+wGW3Wrbopsk+Zokv3lkJgMAOLLGhdlOVdXxSU5N8owkf97df7rwSAAAu7IXdmU+MMlHsroY7RMXngUAYNc2fotZd78pq4vNAgBstL2wxQwAYE8QZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDHLT3A0vrLX85Vn/r00mNwffDn71t6go102zNOWnqEjfRz7/uTpUfYOD/3kO9feoSNdPWHLlx6hD3FFjMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAECPDrKpeWFWv2v54/fkxVfW8qvq7quqqOn2xQQEADqPjlh7gEDwpSW35/GFJHpfk9CQfSvL3C8wEAHDYjQ+z7v78tkV3SvLJ7n7rEvMAABwpI3dlbrV9t2aSX0lyu/VuzAvXy6uqfrqq/qaqvlRV762qM5abGgBg58ZvMdvmSUk+kuTxSb4pyVXr5T+f5JFJfjTJXyd5QJLnV9Vnu/vVSwwKALBTGxVm3f35qrokyVXdfVGSVNUNkzw5yXd091vWL/1wVd03q1C7RphV1VlJzkqSE3PSUZkdAOCfslFhdhDfmOTEJH9YVb1l+fFJLjzQF3T3OUnOSZJT6qZ9oNcAABxteyHM9h8n971JPrrtuSuP8iwAALu2F8Ls/CSXJ7l9d79h6WEAAHZr48Osuy+pqrOTnF1VleTNSU5Ocv8kV693WwIAjLfxYbb21CSfSvKTSX49yReSvDvJs5ccCgBgJ0aGWXc/9kCP15+fneTsbcs6yX9dfwAAbKTxF5gFALi+EGYAAEMIMwCAIYQZAMAQwgwAYAhhBgAwhDADABhCmAEADCHMAACGEGYAAEMIMwCAIYQZAMAQwgwAYAhhBgAwhDADABhCmAEADCHMAACGEGYAAEMIMwCAIYQZAMAQwgwAYAhhBgAwhDADABhCmAEADCHMAACGEGYAAEMIMwCAIYQZAMAQwgwAYAhhBgAwhDADABjiuKUHALg2V1922dIjbKSfueu3LD3CxvnVC1629Agb6d/d7xFLj7CZPnngxbaYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgiOOWHmAJVXVWkrOS5MSctPA0AAAr18stZt19Tnfv6+59x+eEpccBAEhyPQ0zAICJhBkAwBB7Nsyq6seq6q+WngMA4FDt2TBLcrMk/3zpIQAADtWeDbPuflp319JzAAAcqj0bZgAAm0aYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgCGEGADCEMAMAGEKYAQAMIcwAAIYQZgAAQwgzAIAhhBkAwBDCDABgiOOWHgCAw68vv3zpETbOT9zrYUuPsJFec/4fLT3CRjr21AMvt8UMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMMTGhFlV/WRVXbj0HAAAR8rGhBkAwF53WMKsqk6pqpscjvfawe9586o68Wj+ngAAR9Kuw6yqjq2qh1bVbye5KMm91stvXFXnVNWnq+qSqvqTqtq35eseW1WXVtVDquovq+qLVfXGqrrjtvf/6aq6aP3aFyc5edsID0ty0fr3euBuvw8AgCl2HGZVdbeqenaSjyZ5eZIvJvnOJG+uqkry6iS3SfI9Sf5FkjcneUNVnbrlbU5I8pQkj0/ygCQ3SfLft/weP5Dk55P8bJLTkvx1kidvG+VlSX4oyY2SvK6qPlhVP7M98AAANsUhhVlVfW1V/XhVnZfkz5PcNclPJLlldz+xu9/c3Z3kwUnuneSR3f3O7v5gdz81yYeSPHrLWx6X5EfXr3lPkrOTPLiq9s/zE0le1N3P6+4LuvsZSd65dabu/nJ3v6a7fzDJLZP8wvr3/8B6K93jq2r7Vrb9389ZVXVeVZ13ZS4/lFUAAHDEHeoWs3+b5DlJLk9y5+5+eHf/bndvr5r7JDkpyWfWuyAvrapLk9w9yT/b8rrLu/uvt3z+iSTHZ7XlLEm+Icnbtr339s//UXdf0t2/1d0PTvJNSW6R5DeTPPIgrz+nu/d1977jc8K1fNsAAEfPcYf4unOSXJnkMUneV1W/l+QlSf64u6/a8rpjknwqyb88wHt8YcvjL297rrd8/Y5V1QlJvjurrXIPS/K+rLa6vXI37wcAsIRDCqHu/kR3P6O7/3mSb09yaZL/keTjVfVLVfUv1i99V1a7Fa9e78bc+vHpHcz1/iT337bsqz6vlW+pqudldfLBc5N8MMl9uvu07n5Od392B78nAMCidryFqrvf3t0/kuTUrHZx3iXJO6vqXyZ5fZI/TfLKqvquqrpjVT2gqn5u/fyhek6SM6vqiVV156p6SpL7bXvNGUn+vySnJPnBJLft7p/q7r/c6fcEADDBoe7KvIb18WXnJjm3qm6R5Kru7qp6WFZnVD4/q2O9PpVVrL14B+/98qr6+iTPyOqYtd9P8stJHrvlZX+c5Fbd/YVrvgMAwOap1cmU11+n1E37fvWQpccAYGHH3uTGS4+wkV5z/p8sPcJGOvbUD/5Zd+/bvtwtmQAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAY4rilBwCACa763OeXHmEjPfTW9156hA31wQMutcUMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhiiOgQgAAAIdSURBVBBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDHLf0AEuoqrOSnJUkJ+akhacBAFi5Xm4x6+5zuntfd+87PicsPQ4AQJLraZgBAEwkzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAEMIMAGAIYQYAMIQwAwAYQpgBAAwhzAAAhhBmAABDCDMAgCGEGQDAENXdS8+wqKr6TJKPLD3HQdwsycVLD7GBrLeds852x3rbHett56yz3Zm83m7f3TffvvB6H2aTVdV53b1v6Tk2jfW2c9bZ7lhvu2O97Zx1tjubuN7sygQAGEKYAQAMIcxmO2fpATaU9bZz1tnuWG+7Y73tnHW2Oxu33hxjBgAwhC1mAABDCDMAgCGEGQDAEMIMAGAIYQYAMMT/Dxdirl8VG09UAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'esta es mi vida.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A3LLCx3ZE0Ls"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> ¿ todavia estan en casa ? <end>\n",
      "Predicted translation: are you still at home ? <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAJwCAYAAAAjo60MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debiu93zv8c832RmaRJCYghBDacxi10xpSqKDHupQY0IlNdV0aI/TKqo4xqI4pGoeiiiJqhBFqdKKlCKmIIKIJKSJEIkk3/PH/exmrWXvyJa99/1be71e17Wv/az7edZa33VfyXre+x6ruwMAwPx2mHsAAAAmwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwmwAVfXLVfWhqrrp3LMAAPMRZmM4JMldkjxs5jkAgBmVm5jPq6oqyUlJjk3yO0mu3t0XzjoUw6iqqyXZeemy7j55pnEA2MpsMZvfXZNcLsljk1yQ5DfnHYe5VdXlq+r1VXVuku8k+caKPwBsp4TZ/B6S5Mju/nGSt2barcna9oIkN0/yP5L8JMkDkjw5ybeT3G/GuQDYyuzKnFFV7Z7ku0l+q7s/VlW3SPKJTLszz5x3OuZSVd9Ocv/FfxNnJzmgu0+sqvsneVh3323mEQHYSmwxm9fvJTmjuz+WJN39mSRfTfL7s07F3K6Q5JuLx2cl2Xvx+BNJbj/LRACrXFXtXlUPqarLzz3LJRFm83pwkjetWPam2J251n0tyXUXj7+Y5PcXJ4ncO8kPZpsKYHW7b5LXZnrvHZZdmTOpqn0zHci9f3d/dcnya2Y6S/NG3f2VmcZjRlX1hCQXdvdLq+rXk/xDkp0y/UPqcd39slkHBFiFquojSa6S5MfdvX7mcTZJmMHgqupaSdYn+Wp3f27ueQBWm6raL8lXktw6ySczHbt7wpwzbYpdmTOqqmstdlFt9LltPQ9j6u6Tu/vvRRnAL+zBST62OJb7HzPwIUO2mM2oqi5Msk93n7Zi+d5JTuvuHeeZjG2tqp6Y5BXd/ZPF403q7hdto7EAtgtV9dUkz+ru11XVvZO8NMm+PWAECbMZVdVFSa7a3aevWH7tJCd09+7zTMa2VlXfSLK+u7+/eLwp3d3XvYTnAViiqm6f5AOZ3m9/VFU7Jzk1yf26+9h5p/tZ6+YeYC2qqpcuHnaS51TVj5c8vWOmfeCf2eaDMZvuvs7GHgNwmR2S5Kju/lGSdPf5VfX2JIdmuh3iUITZPG66+LuS7J/k/CXPnZ/k+ExXf2cNqqqbd/dn554DYLWrql0yXSbj/iueelOS91fVHt19zrafbNPsypzJ4qD/t2e6kvsP556HcSx2cX8hyRuTvLW7vzXzSACrUlVdKdM9qN/U3ReteO5BST7Y3afOMtwmCLOZVNWOme6DePNRT9llHlV1gyQPzPQvvOsm+VimSDuyu8+ec7a5VNWuSR6X5MBM1yFadkZ5d99sjrkAtjRhNqOqOjHJfRan78LPqKrbZIq0+ybZM8k/dPd9551q26uq1yS5V5J3JDkl0/GZ/627nzHHXABbmjCbUVUdkmmryIO6+4y552Fci0B7ZZKbrcXLqFTVD5Lct7s/OPcswPgWZ7dfqsAZ7Ux3B//P60lJrpPkO1X17SQ/Wvqk3TNrW1VdN8kDMm0xu36mXZoPn3Wo+fw4iWPtgEtr6a3r9kjyxCT/nuQTi2W3y3QFhBdu47l+LlvMZlRVT7uk5+2eWZuq6tGZYuw2ST6f5M1J3tzd35l1sBlV1WOT3DjJI1cewAtwSarqdUm+0t3PXrH8KUlu3N0PmmWwTRBmMJiq+laStyZ5o9swTarqPUnulOSsJCck+enS57v7nnPMBYyvqs7OdG/ME1csv36S47t7z3km2zi7MmE81xrxNiEzOyPJu+YeAliVfpTkLklOXLH8LpkOkxiKMJvR4rYQf5rpBIBrJdlp6fNr8SBvpnsuJUlVXT3Tfxc7r3j+o3PMNafufujcMzAuv0v5Of4qycuran2STy6W3TbTHQGePtdQmyLM5vXMJPdL8pxM/+E8Ocl+SX4/yVPnG4s5LYLsrZl23XWmO0Qs3YLmTQaW87uUTeru51XVSZmuhbjhckNfTHJId799tsE2wTFmM1qczvvI7j6mqn6Y5Bbd/bWqemSSA7v7PjOPyAwW93DbO8mjk3wqycFJrprkL5I8YcSb7m4LVfXQXLxFZOVWxKFOd2fb8ruU7ckOP/8lbEVXzXQgc5Kck+QKi8fHJLn7LBMxgl9L8ifd/aVMW8pO7+6/T/InmbYMrDlV9eRMp7V/OtOWkHdnOmN1rySvmW8yBuF3KZdKVV2hqvZa+mfumVYSZvM6OcnVF49PTHLQ4vHtkpw7y0SM4JcyHeyeJD/IdAuiZHrjWavXtjssyeHd/ZRMZ2S+bHEm5guTXHvWyRiB36VsUlVdu6reV1U/SfL9JKcv/pyx+HsojjGb17sy3fvvk0lekuStVXVYkmskef6cgzGrLyX5lSQnJflMkkcsLqHx6CRr9Vpm18x0cchkeqPdcHr7WxfLD5tjKIbhdymX5LWZtqI+LBu5pdtoHGM2kMVtd+6Q6UJ4/zD3PMyjqh6YZKfufl1VHZBpd8zeSc7LdLDqO2YdcAZV9fVM95U9vqo+leQ13f3/qurgTBff3XvmERlIVd02ye3jdylJquqcJLft7s/PPculIcxmVFV3TvKv3X3BiuXrktx+LV4WgZ9VVbtl2oJ28lq9p2pVvTrJt7v76VX1iExn3n0yyQFJ3t7dtpgBG1VVn0tyaHd/eu5ZLg1hNqOqujDJPt192orleyc5zbV3YFJVOyTZYcM/YqrqfllsXU7yqu7+6SV9Ptu3qrpvkv/q7g8sPv7zJIcn+UKmN+Tvzjkf86qqX0/yv5M8auXV/0ckzGZUVRcluWp3n75i+Q2SHDfabSLYeqrqUp9Z2N0P25qzjKiqrpXkWyvviFBVlWTf7j55nskYQVWdkOTx3f2Bxe7/f03y55kuNXNqdz9g1gGZ1eISKrtkugbkeUmW7aUa7b3Wwf8zqKqjFw87yZuq6rwlT++Y5CaZfrGwdlx5xcd3TnJRkg33yrxJprOo1+ru7W8k2SfJaSuW77V4ztblte3aSb68eHyvJO9eXFT0A0neP99YDOIxcw+wOYTZPL6/+LuSnJnlp3Ofn+RfkvzNth6K+XT372x4XFVPyfTfxEO7+0eLZbsn+dtcHGprzcq7H2ywR5KfbONZGM9Pklxu8fjAXHxtu7OWLGeN6u7Xzz3D5rArc0ZV9bQkL9jw5gtJUlXfzXS18hNWLL9xkn/q7qvNM9m2V1UvXTx8dKZT3pfecHjHJLdOcn5332Fbz8Y4qurdma7/9y+ZbsG0X3efUlUHJXlpd99w1gGZXVVdNcmDk1wvyVO7+4yqukOSU7r7G/NOt5wLzM7rmVmytayqrlZVD6+q2884E/PbIxdfLHOpfZLsto1nmdtNF38qyf5LPr5pkusnOT7JoXMNxzAek2lvw32SPKK7T1ksv0fsylzzqupWmXZ1PzDJH+Ti6yDeLcmz5pprU2wxm1FVvS/JMd39kqraI9OFRXfP9Mb8B939hlkHZBZV9bpMu2OenOmSEEly2yTPTfLh7j50nsnmU1WvTfK47j577llGsbiMyi0y3Rli2T+yF7fwApJU1YeTfLS7n7Y4EeDm3f31qrpdkr/r7qHuHiLMZlRVp2XaZfW5qnpIptN5b56p6p/Y3Wv19jtrWlX9UqZbDT0syU6LxRdkOsbsSd3940197lqxWEd3SPLV7v7m3PNsa1X1G5nuerCxC+u2S+3Axarq7Ew3tv/6ijDbL8mXunvXWQdcwa7MeV0uyX8tHt89ybsW12P6UKb94KxB3X1udz8q05vuLTNdRHWv7n7UWo2yqnpdVT1q8XjnTLdh+kCSL1fVPWYdbh4vSfLeJNfs7h1W/FlzUVZVO1fVM6rqK1X1k6q6cOmfuedjducmueJGlv9KfvZM79kJs3mdnOQOizPuDkpy7GL5Xll+kDNr04WZLplxweLxWnZQLt6te89M/6i5WpKnL/6sNfsleeaSY6nWumcmOSTTluaLMh0G8PJMZ8A/asa5GMNRSZ5WVbssPu7F1rLnJnnnXENtijCb14uSvDHJtzPdnHrDNarunLV7WYQ1r6rWVdXzM11K5bOZ/ls4s6qeV1U7XfJnb7eumIv/ZXtwkncu7pjxd0luNNtU8/l4EmcaXuy+mQ76f1Wmf8Qc1d2PTfK0TAd4s7Y9KdMGj9MznUD1L0lOzHQ5lT+bca6Nch2zGXX3q6rquCTXSnJsd1+0eOprmU75Zm16XpL7J3lEpl8gSXKnJM/J9I+pJ80015xOTXKTxaVEDsp0u51kOlFmLd6O6ZVJXlBVV88U7svWQXcfP8tU87lqkg2XlzknyRUWj4/JtFWENWxx0tAdF7dmOiDT79Hju/uD8062ccJsJlV1+SQ36+6PJVl5Y9X/ysW/ZFh7HpDkYd39j0uWfa2qTk/y6qzNMHtNkrclOSXTFpF/Wiy/TaazmdeaIxd/H7GR5zpr704IJ2e6xMzJmbaEHJTp9+rtsvwC3qwxS99ru/tDmY7h3vDcHZKc0N1nzjbgRgiz+VyU5H1VdVB3f3zDwqq6Rab/cK4x22TM7fKZtpqu9LVcvCVgTenuv6iqz2e69c7bu/v8xVMXZG1uEbnO3AMM5l2ZLjHzyUwnRry1qg7L9Hv0+XMOxuxW3XutY8xm0t0/zHRA4kNWPPWgJO/v7jO2/VQM4rNJHruR5Y9L8pltPMtIzk3yG0mOrap9F8t2zrTrak1ZXCLkRpkOcH9fkosWy+6W6cK7a0p3P6W7n7V4fGSSOyb56yT37u4/nXU4ZrUa32uF2bzekOR/bjigu6p2yLQb63VzDsXs/jjJIYtT/1+/uFTElzP9InnyzLPNoqoemOTtSb6SaWvRhpMgdsi0vtaUJevjq1m+PnbM2lwfz6qqR2z4uLv/rbtflOSaVfXMGUdjDKvqvVaYzevYTJfF2HAD6wMzbQF4z2wTDWzxP9NacFKSGyR5R6aD2/dcPL5hpmNo1qI/TnJYdz8h0+7LDT6Z6er3a431sdyDk/zHRpZ/Oj+7pWS7VlW/XVWPXNwbksmqeq9dK290Q1qchfnmXPyL48FJ3ra4yCwrLDlrdXv3jSQXdPefdvfvdfe9u/vPkpy3eG4t+uUkn9jI8nNy8X3v1hLrY7mrZLoUwkrfz3TG5ppQVf870/F2f5bkP6vqpjOPNITV9l4rzOb3hiQHL46ZuVeS1888z2yq6sNV9dqquuLi8dFVdcjcc82gMp1Zt9IeSX6yjWcZxSmZtiKudOds/ESJ7Z31sdzJmS4ps9KdM10ncq14VKb7LF8j00kQx1bV3avqWovrI+5TVdeaeca5rJr3Wmdlzqy7v1BVn0vyliTf7u5/n3umGX0+0/Wqfrp4fLkkL6+qWy0uFrldq6qXLh52kudU1dK7P+yY5NZZuwf/H5HkpVX18MXH+1bVnTJd8+3ps001H+tjuVcl+avF7bo2XA7hwEzX/ltLZ+3ulcWFyrv72YvDP963eO5XM201ukHW3uVUVtV7rTAbwxuTvDjJmj57qLv/aMmHf5QkVfXXSY5Z3D7jyO5+wwyjbSsbdjtUkv2TnL/kufOTHJ/kBdt6qBF09/MW1yM6NsmuST6cadfuC7r75bMONwPrY7nufmFVXSnJSzMdO5RM/8+8pLufN99k29xXMp2te1KSdPdfVtUrMt3C64uZduXtNtdwA1gV77XVvbE9JmxLVbVXphB5VXefOvc8o6mqGyR5RZL13b3dX8erql6b5HGLq1WzRFXtlumNZ4dMF4Zcc5fKWMr6WG5x3+EbZfrHzZpbH1X1mCR37e7fm3uWEa2W91phBgAwCAf/AwAMQpgBAAxCmA2iqg6fe4aRWB/LWR/LWR/LWR/LWR/LWR/Ljb4+hNk4hv4PZQbWx3LWx3LWx3LWx3LWx3LWx3JDrw9hBgAwiDV/VubOtUvvmt3nHiM/zXnZKbvMPcYwrI/lrI/lrI/lrI/lrI/lrI/lRlkfP8yZZ3T3lVcuX/MXmN01u+c2deDcYwCsflVzTzCWNb7hg0v2wT7ymxtbblcmAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCBWfZhV1U5zzwAAsCUMF2ZVdXBVfayqzqyqH1TV+6tq/8Vz+1VVV9X9q+pDVXVukj9cPHf7qvrnqvpxVX2nqv5fVe056w8DALAZhguzJLsneXGSWye5S5KzkrynqnZe8prnJHlFkhsleXdV3TTJB5IcneTmSe6d5BZJXrPtxgYAuGzWzT3ASt39zqUfV9VDk5ydKdS+vVj819195JLXPDvJ27r7hUuWPTLJf1TVVbr7tBVf8/AkhyfJrtltq/wcAACba7gtZlV1vap6S1V9rarOTvK9THNea8nLjlvxabdK8qCqOmfDnyQfXzx3vZXfo7uP6O713b1+p+yyNX4MAIDNNtwWsyTvSfKdTMeOfSfJBUlOSLJ0V+aPVnzODkleneSvNvL1vrMVZgQA2OKGCrOq2jvJ/kke3d0fXiw7ID9/zuOT3Li7T9zKIwIAbDWj7co8M8kZSQ6rqutX1a8leWWmrWaX5LlJbl1Vr6yqWy4+97er6lVbe2AAgC1lqDDr7ouS3C/JzZJ8PsnLkzw1yXk/5/P+M8mdk+yX5J+TfDbTmZvf24rjAgBsUUPtykyS7v5QkpusWLzHkse1ic87LsnBW2suAICtbagtZgAAa5kwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYxLq5B5hdJbXOathghytece4RhvL9g68/9whDOf3A8+ceYSg3OOxzc48wlB0uf7m5RxjKhd//wdwjjKV77glWBVvMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYxe5hV1UOq6vtVtcuK5W+uqqMXj/+wqk6sqvMXfx+24rVdVfdZseykqnrS1v8JAAC2jNnDLMk7Ms3xuxsWVNXlk9wryd9W1b2SvCzJi5PcJMlLkryiqn5nhlkBALaadXMP0N3nVtWbkzwsydsXix+Q5Owk703yz0ne2N0vWzz3laq6VZI/SfKeX+R7VtXhSQ5Pkl2z22WYHgBgyxlhi1mS/E2Su1XVNRcfPyzJ67v7giT7J/n4itf/S5Ib/aLfrLuP6O713b1+p+V7UAEAZjNEmHX3Z5Mcn+TQqrpJkvVJXrP0JRv7tBWPa8XzO23RIQEAtrIhwmzhb5IcmuThST7e3V9eLP9ikjuueO0dk5yw5OPTk+yz4YOquurSjwEAVoPZjzFb4q1JXpTkkUkesWT585O8o6o+neQDSQ5O8sAk917ymg8leXRV/WuSC5M8O8lPtsXQAABbyjBbzLr7h5kO/j8/F58EkO5+d5I/SvKETFvJHpfkUd299MD//5Xk60k+kuTIJK9Octo2GRwAYAsZaYtZMu1+/Lvu/tHShd39yiSv3NQndfcpSe6xYvE7t/x4AABbzxBhVlV7JfmNJHdPcvOZxwEAmMUQYZbpjMy9kvyf7v783MMAAMxhiDDr7v3mngEAYG7DHPwPALDWCTMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEGsm3uA2XXSF1ww9xTDuPD00+ceYShX+uiuc48wlO/f4hpzjzCUC29747lHGMoP99ll7hGGsseRn5p7hLH0hXNPsCrYYgYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwCGEGADAIYQYAMAhhBgAwiFUbZlX1kap62aX9GABgdOvmHuDnqapDk7ysu/dY8dS9k/x0208EALB1DB9mm9LdP5h7BgCALWmYXZlVdeeq+mRVnVNVZ1XVv1XVY5K8NsnuVdWLP09fvN6uSgBguzLEFrOqWpfkqCR/m+SBSXZKckCSLyR5fJJnJ7ne4uXnzDEjAMDWNkSYJdkzyRWSvKe7v7ZY9qUkqapbJunuPnVLfbOqOjzJ4Umya3bbUl8WAOAyGWJX5uJ4sdcleX9VvbeqnlhV+27F73dEd6/v7vU7ZZet9W0AADbLEGGWJN390CS3SfLRJPdM8pWqOmjeqQAAtp1hwixJuvuz3f3c7r5Lko8kOSTJ+Ul2nHMuAIBtYYgwq6rrVNX/rarbV9W1q+quSW6W5IQkJyXZtaruVlVXqioHhQEA26VRDv7/cZIbJHlHkisl+V6SNyd5bnf/tKpemeStSfZO8owkT59pTgCArWaIMOvu72W6kv+mnn9kkkeuWHaXzfkYAGB0Q+zKBABAmAEADEOYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADGLd3APAyC745rfmHmEoN3jGWXOPMJQv/t9fmXuEoez+Tf/WX+oKV7nS3CMM5YLvnTb3CGPpjS/2fxEAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAgtrswq6r9qqqrav3cswAAbI7tLswAAFarVRlmVXVwVX2sqs6sqh9U1furav/F099Y/P2pxZazj8w0JgDAZlmVYZZk9yQvTnLrJHdJclaS91TVzotlSXJwkn2S3HuOAQEANte6uQf4RXT3O5d+XFUPTXJ2pij79mLx97v71I19flUdnuTwJNk1u23FSQEALr1VucWsqq5XVW+pqq9V1dlJvpfpZ7nWpfn87j6iu9d39/qdsstWnRUA4NJalVvMkrwnyXeS/OHi7wuSnJBk5zmHAgC4LFZdmFXV3kn2T/Lo7v7wYtkBufhnOX/x944zjAcA8AtbdWGW5MwkZyQ5rKq+leQaSZ6faatZkpyW5NwkB1XVSUl+0t1nzTEoAMDmWHXHmHX3RUnul+RmST6f5OVJnprkvMXzFyR5bJKHJzklyVHzTAoAsHlW4xazdPeHktxkxeI9ljz/6iSv3qZDAQBcRqtuixkAwPZKmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADEKYAQAMQpgBAAxCmAEADGLd3AMAq8eFZ5899whDueFjj597hKHsuO815h5hKL/74c/NPcJQjrr7AXOPMJaTN77YFjMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQWzRMKuqj1TVy7bk1wQAWCtsMQMAGIQwAwAYxNYIsx2q6tlVdUZVnVZVL6iqHZKkqq5YVa+vqjOr6tyq+mBV3XjDJ1bVoVV1TlXdo6q+VFU/rqqjq+ryVXWfqvpqVZ1VVW+sql9a8nlVVX9cVV9bfN3PVdWDtsLPBgCw1WyNMHtgkguS3D7JY5I8Psn9Fs+9Lsltkvxuklsn+XGSY5ZGVpJdkvyvxdc5MMn6JEcmOSTJ7yX5H0l+O8mjlnzOXyb5gySPTnKjJM9J8qqq+q2NDVhVh1fVcVV13E9z3mX8cQEAtox1W+FrntDdf754/JWqOizJgVV1XJJ7Jvm17v5oklTVg5OcnCnCXr1kpkd395cXr3lLkickuWp3n7FYdlSSuyZ5YVXtnuSJSe7e3R9bfI1vVNWtM4Xae1cO2N1HJDkiSfasvXqL/vQAAL+grRFm/7ni41OSXCXJ/kkuSvKJDU9091lV9blMW7k2OG9DlC18L8mpG6JsybINn3OjJLtm2vK2NLJ2SnLSZfg5AAC2qa0RZj9d8XFn2mVal/A5S4Pqgo08t6mvmSV//06mrW+XNAsAwLC2RphtygmZIup2STbsytwzyU2TvPYyft3zkly7uz90WYcEAJjLNguz7v7q4tiwV1XV4Un+K8mzkpyd5C2X4ev+sKpekOQFVVWZom+PJLdNctHieDIAgOFt6+uYPTTJvyc5evH3bkkO7u5zL+PXfWqSp3oYziQAAAmMSURBVCd5UpIvJDk20xmc37iMXxcAYJup7rV9UuKetVffpg6cewxgFap12/JokPHtuO815h5hKL/7j5+ae4ShHHX3A+YeYSjHnPziT3f3+pXLXfkfAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDr5h4AYLXqCy6Ye4ShXHDSyXOPMJSj7rF+7hGG8sSPvHfuEYZyzHU3vtwWMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEEIMwCAQQgzAIBBCDMAgEFsV2FWVY+pqv+oqh9V1beq6ilzzwQAcGmtm3uALezAJH+e5AtJ7pzk1VX1he4+et6xAAB+vu0qzLr7Xks+/HpVPTvJvnPNAwCwObarMFuqqv5Pkp2S/P1Gnjs8yeFJsmt228aTAQBs3HZ1jNkGVfVnSR6f5G7d/d2Vz3f3Ed29vrvX75Rdtv2AAAAbsd1tMauqvZP8RZLf6u7PzD0PAMCltT1uMdsvSSX54sxzAABslu0xzL6Y5FeTnDL3IAAAm2N7DLObJHlTkivPPQgAwObYHsNstyQ3zHRGJgDAqrHdHfzf3R/JdIwZAMCqsj1uMQMAWJWEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAINbNPQAA24nuuScYyoXf/d7cIwzlwF+6cO4RVgVbzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGIcwAAAYhzAAABiHMAAAGsWrCrKqeVFUnzT0HAMDWsmrCDABge7dFwqyq9qyqK2yJr7UZ3/PKVbXrtvyeAABb0y8cZlW1Y1UdVFVvSXJqkpsvll++qo6oqtOq6odV9c9VtX7J5x1aVedU1YFV9fmq+lFVfbiqrrPi6/9xVZ26eO0bkuyxYoTfTHLq4nvd4Rf9OQAARrHZYVZVN66q5yU5OcnbkvwoycFJPlpVleS9Sa6R5LeT3DLJR5N8qKr2WfJldknylCQPS3K7JFdI8sol3+O+Sf4yydOSHJDky0meuGKUNyd5QJLLJTm2qk6sqj9fGXib+BkOr6rjquq4n+a8zV0FAABbRXX3z39R1d5JHpjkIUluluSYJG9McnR3n7fkdb+e5OgkV+7uc5cs/0ySt3T386rq0CSvTfIr3f3lxfMPXCzbtbsvqqp/TfKF7j5sydf4YJLrd/d+G5nvckn+Z5IHJ7lTko8neX2St3f3OZf0s+1Ze/Vt6sCfuw4AYHPULrvMPcJQjvnGv809wlB23OfET3f3+pXLL+0Wsz9K8pIk5yX55e6+Z3e/Y2mULdwqyW5JTl/sgjynqs5JcpMk11vyuvM2RNnCKUl2yrTlLEn2T/KJFV975cf/rbt/2N2v6e67JvnVJFdJ8rdJ7nMpfz4AgNmtu5SvOyLJTzNtMftCVb0r0xazf+ruC5e8bock38u01Wqls5c8vmDFcxs22/1Cx7xV1S5JfivTFrPfTPKFJI9PctQv8vUAAOZwqUKou0/p7md19w2T/EaSc5L8XZJvV9ULq+qWi5cen+SqSS7q7hNX/DltM+b6YpLbrli27OOa3LGqXpXp5IOXJTkxya26+4Dufkl3n7kZ3xMAYFabvYWquz/Z3Y9Msk+mXZw3SPLvVXWnJB/MdHzXUVV1j6q6TlXdrqqesXj+0npJkkOq6rCq+uWqekqS26x4zYOSfCDJnknun2Tf7n5yd39+c38mAIARXNpdmT9jcXzZkUmOrKqrJLmwu7uqfjPTGZV/k+lYr+9lirU3bMbXfltVXTfJszIds3Z0khclOXTJy/4pydW6++yf/QoAAKvPpTorc3vmrEwAtgZnZS7nrMzlLutZmQAAbGXCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEMIMAGAQwgwAYBDCDABgEOvmHgAAtkd93nlzjzCUg65+i7lHGMyJG11qixkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAg1s09wByq6vAkhyfJrtlt5mkAACZrcotZdx/R3eu7e/1O2WXucQAAkqzRMAMAGJEwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYhDADABiEMAMAGIQwAwAYRHX33DPMqqpOT/LNuedIcqUkZ8w9xECsj+Wsj+Wsj+Wsj+Wsj+Wsj+VGWR/X7u4rr1y45sNsFFV1XHevn3uOUVgfy1kfy1kfy1kfy1kfy1kfy42+PuzKBAAYhDADABiEMBvHEXMPMBjrYznrYznrYznrYznrYznrY7mh14djzAAABmGLGQDAIIQZAMAghBkAwCCEGQDAIIQZAMAg/j+yaKWo/aX/HgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "translate(u'¿todavia estan en casa?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DUQVLVqUE1YW"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input: <start> trata de averiguarlo . <end>\n",
      "Predicted translation: get out of figure it out . <end> \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbEAAAKICAYAAAD+TcRJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZhkdX3v8feHmXGQTYMLoldFUFBxAR1RIqJIXEly4/IYTeJG4qjRiNeLIWoSt+CCYNSLGokRXOMWvS54NS64b0FxCyqioqCsisigzODwvX+c09I2PTA0XefUr/r9ep55uvpUdfW3YKbffU6dJVWFJEkt2mbsASRJWiojJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZq8ceQFLbkmwL3Boo4PtVdenII2kFcU1M0pIkWZ3kZcCFwNeBbwIXJjkqyZpxp9NK4ZqYpKU6CngU8CTgs/2yewEvpvsF+fCR5tIKEs+dKGkpkpwDHFpVH1qw/BDg9VW16ziTaSVxc6Kkpboe8P1Fln8fuP7As2iFMmKSlurrwNMWWX4Y8LWBZ9EK5eZESUuS5EDgQ8BPgS/Q7Z24P3BT4EFV9dmr+HJpWRgxSUuW5KbAU4DbAgFOBV5TVT8ddTCtGEZMktQsd7GXtNWS3GVrH1tVX53kLBK4JibpGkhyOd17X7mah1ZVrRpgJK1wrolJuiZuNfYA0nyuiUm6xvrTSh0JvLqqfjT2PFq5jJikJUmyAbhDVZ0x9ixauTzYWdJSfQS479hDaGXzPTFJS/Vx4EVJ7gR8Bbhk/p1V9Z5RptKK4uZESUvS76m4Je6dqEEYMUlSs3xPTJLULN8Tk7RkSXYGHgjcArjO/Puq6gWjDKUVxc2JkpYkyT2AE4GNwI2AnwC79p+fUVV3GnE8rRBuTpS0VC8D3grcDLiUbnf7WwAnAy8dcS6tIK6JSVqSJBcBd6uq05L8Ati/qr6d5G7A26rqNiOPqBXANTFJS7Vp3u1zgVv2tzfQXRhTmjh37JC0VF8F7gacBnwS+KckuwB/AXxjxLm0grg5UdKSJFkH7FhVJyW5EfAm4J50UXt8VX1z1AG1IhixESS5DfA64DD/oUvS0vme2DgeC9wHOHTkOSSpaa6JDSxJgDOAjwJ/BNy0qjaPOpS0BEm+SXeV50V5nJiG4I4dwzsI2BF4GvAg4MHAB0adSFqady/4fA2wD937Yq8efhytRK6JDSzJCcCmqlqf5Ghgt6p6+MhjScsmyTOBW1bVU8eeRbPPiA0oyfbA2cAhVfWZJPsAX6DbpHjhuNNJyyPJHsDJVfV7Y8+i2eeOHcN6GHBBVX0GoKq+BnwPeOSoU0nL60DgV2MPoeWVZPskj0lyvbFnmc/3xIb1aOAtC5a9hW5vxdcOP460dEnev3AR3QmA9wWeP/xEmrBHAK8HDgOOHXmW33Jz4kCS3Bz4IXC7qvrevOX/g25vxdtX1WkjjSddY0mOX7DocuB84BNV9Z8jjKQJSvJJ4MbAr6pq3cjj/JYRkyRdpSS70Z2JZT/gi8BdqurUMWea43tiA0pyi/44sUXvG3oeSdpKjwY+07+P/yG6t0CmgmtiA0qyGdi1qs5bsPwGwHlVtWqcyaRrLskPWfxg56K7vtjpwL9V1cL3ztSYJN8DjqyqE5I8FHgVcPOagoC4JjassPg/+h3o/tFLLTke2JluD9u39H++1y97P7AZeE8S975tWJLfp9th5139og8C2wF/MNpQ87h34gCSvKq/WcCLk8zf/XgV3Xbmrw0+mHTt7A68pKpeMn9hkr+l21HpoUmeDRwBvH2MAbUsHgu8r6ouAaiqTUneCTyO7vR5o3Jz4gCSnNTfvDfdwc3zLya4iW7vxKPn77UoTbskv6R7g//0BctvDXy1qnZKshfwlaraYZQhda0kWQucAzyqqj48b/kBwEeAXapqw1jzgWtig6iqg/odOt4JHFpVF489k7QMfgXci+69r/nuxRUHO68Cfj3kUFpWO9IdF/Y7h0xU1WeTPJHurZBRI+aa2ECSrKJ73+vO07JrqnRtJHkW8I/AG4D/ottcvh/dZqYXVtVLkjwDeFBV3W+0QTXTjNiAkpwOPLzfTVVqXr/TxtOA2/aLvgO8sqre0d9/XaCqyh2XNBFGbEBJHgs8CviLqrpg7HkkaTFXcfjElVTV7hMe5yr5ntiwDgduBfwkyVnAJfPv9CKCkqbE/HMj7gA8A/gy3Y5pAPvTbTo+ZuC5rsSIDWvhRQSlpvR7JO5eVRckuZirvrLzTsNNpuVUVb+NU38NxJdW1YvmP6Z/T3TvgUe7EjcnamKSHES3+fQWwHXm31dV9x1lKF0r/Sbxt1fVxv72FlXVGwcaSxO0NYdSjDNZxzUxTUSSxwH/ArwXuA/wPmBPus2pCy9Ho0bMhSnJaroz1n+pqn427lSasEvo/g0vPJTiPkzBdeOM2ICSXAd4DlesnayZf/+MnTvxcOCpVfX6frPTs6rqB0mOZeTjSnTtVdVvkryHbq9EIzbb/hl4dZJ1dGewB7gH3Zk8njfWUHM8d+KwXkj3P/4YumsvPRN4Nd0Pgb8eca5J2B34WH97I92bw9C9Yfy4MQbSsvs6cOuxh9BkVdVRdGexvyPw8v7PHYHHVtVLx5wNXBMb2iOAJ1XVh5McTXc+su8n+TZwP+B14463rH5Gd7Q/wE+AOwDfAG4AXHesobSsngcck+S5wFe48t62Px9jKC2/qnon3RmHpo4RG9YuwNzZOjYA1+9vfxgY/TeaZfYZ4P7AN+n+8r8qyf2Ag5mCk4ZqWZzYf3wPv7uX4tzVGmZp87iAJNdnwRa8sX9ZMWLD+jFw0/7j6cAD6H6D3Z/ZO7/cU4Ft+9svBn4D3JMuaP801lBaVgeNPYAmL8kt6XbSOojffR9/Kn5ZcRf7ASV5MbChqo5M8nDg34GzgJsBL6uq54w6oCQtkOQTdFuNjgZ+yoJjA6vqU2PMNceIjSjJ3enWTk6rqg+OPc9y8irWK0OSOwJPBPagu0LD2Un+BPhRVZ0y7nRaDkk2APeoqm+NPcti3DtxQEkO7I+vAaCqvlRVLwc+nOTAEUebhGxh+Vp+93pqalSS+9Odvf5mwH25YoedPYDnjjWXlt0P6f7dTiXfExvWSXSX+T5vwfLr9fc1v3bSX3oDuk0OT+p/i5uziu5aU98ZfDBNwguBZ1TVa/pjAed8Evjf44ykCTiM7or0f73wrB3TwIgNa+6N0IVuwILdkxv2N/3HAH8FbJ5339xVrJ808EyajL2BDy2y/OfAzgPPosl5H92a2HeTbKTbSeu3PO3UCpDk/f3NAt7S/0WYs4ruGKrPDz7YBFTVrQCSnAQ8tKouHHkkTc6FdJsSz1iw/C50OyxpNjx17AGuihEbxtxpeUL3D3/+7vSbgM8C/zr0UJNUVe5+PfveBrwsySPofkFbneTedHuxHT/qZFo2034iZ/dOHFB/ZoOjq2pWNh1epSR7Ag9n8bPYHzrKUFo2SdYAJwCPpPsF7fL+49uAx1XV5i1/tVqSZBe6U0/tAfxDfymeewI/raofjjqbERtOkm0Aqury/vObAH8InFpVM7E5cU6SQ4D/AE4B7kq3F9sedNvWP1NVfzzieFpGSfYA9qXb2/mUqvreyCNpGSW5K/Bxur0U9wZu25/M+3nAnlX1Z2PO5y72wzqRfseHJDsAJwMvAz6V5DFjDjYBLwCeX1X7050A+NHAbnQnBf7keGNNVpI7Jjk2yf9Lsmu/7E+S7Dv2bMstyf9Msrqqvl9V766qdxqwmXQ08Mqq2pfu3/Kcj9Ad5zoqIzasuwKf6G8/FPglcGPgCXSXLpklewHv6G9fBmxXVZfSxe3po001QSvwuKl/B85J8tokvz/2MJqYuwKLvS92Nt35YEdlxIa1I/CL/vb9gfdW1WV0YdtjtKkm42KuOHfi2VxxyY7VwO+NMtHkzR039RB+94DuTwL7jTLRZO1CdzmhWwOfTvKDJC9MstfIc2l5/ZrF/83elisf8zo4IzasHwP3TLI93cl/587mvjNTcIXUZfYl4ID+9olcccmO44EvjDbVZK2o46aq6uKqOr6q7gfcnO5acQ8CTk3y5XGn0zJ6H/DcJHNn7agku9FdeeM/xhpqjhEb1suBN9MdQ/MT4NP98gPpLlkyS57BFVeBfR7wn8DD6M7e/1cjzTRpc8dNLTTzx01V1dl0EXsx3XXj7jruRFpGh9P9EnY+sB3dIUGnAxcBfz/iXIB7Jw6u39PnFsBHq2pDv+wQ4BdV9blRh1sm/fkh7w98qapWzKXrk7yU7rRaj6C7btw6utOMnQAcX1UvGG+6yUlyEPDndL+kALwXeHNVnTTeVFpuSe5L9wvZNsBXq+pjV/MlgzBiA0lyPeBOVfWZRe67J91u9jNzdoskl9LtinvG2LMMZQvHTW0DvJUZPG4qycvoXuuN6fZUewvd1co3XuUXqhkt/NwyYgNJsiPdDg4PmL/GlWQfuvePblZVF4w133JL8iXgOdPy29qQkuzOFb+xzuxxU0k+Txeut499dV9NRgs/t4zYgJK8le6imE+ct+xougMGZ+rg3yQPAl5Ct2v5V1hwguNZ+aGX5A1b+9hZPEtJv+l4PxY/K8ubRhlKy2raf24ZsQEleQDdsTW7VNVl/Rk8zgKeWlXvGXe65ZXk8nmfzv9LFqBm5aKYST6wYNGBdJsR53bUuQPdGtmnp+Ef/HLqd6X/ALA73f/XzXSHUFwGbBz77OZaHtP+c8sTAA/ro3S70v8R8B7gYLrfXhf+IJwFjwfO5HcvxQLdD/RbDD/OZFTVH83dTvIsumNqHj93fsz+cIp/Y/b2PgV4JfBVulNOnQPsQ3dtvNcyBXutadlM9c8t18QG1u/BtldV/UmSNwEXV9VTxp5ruSXZDOxaVectWH4D4LxZWRObL8nZwMFVdeqC5XsDH6+qm4wz2WQk+Rlw76r6VpKLgP2q6rv9mez/T1XdaeQRtUym+eeWa2LDexPwlSQ3Bx5C91vNLNrSBUB3AC4deJah7ADclG73+vl2pTu+ZtaEKw7SP5/uGLnv0m1quvWWvkhNmtqfW0ZsYFX130m+SXe5irOqaqbObJDkVf3Noruk+fwzkayi2wnga4MPNoz/AI5P8kyuOND7HnRnNhj9vYMJ+BZwZ+AHwJeBI/o18CfQHQyrGTHNP7eM2DjeDLwCeM7Yg0zAHfuPAW7H755DcBPdeyhHDz3UQJ4MHEN3rNiaftlv6N4Tm7UTPAMcCWzf3/574IPAScAFdAd8rxhJvg3cpqpm+WfqVP7c8j2xESTZme6SLK+rqnPGnmcSkhwPHFZVvxx7lqH1O3PsQRfy01fKRVDht3+3L6wV9oMlyVOBG1TV88eeZVKm9eeWEZMkNcsTAEuSmmXEJEnNMmIjSbJ+7BmGttJes6939q201zyNr9eIjWfq/jIMYKW9Zl/v7Ftpr3nqXq8RkyQ1a8XvnXidrK1tf3uoy3AuYyNrWHv1D5whK+01+3pn30p7zWO93ku5hE21MYvdN8sH5m2Vbdmeu2dqzqAiSVrgS/XxLd7n5kRJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmzVzEktwnSSW54dizSJIma+YiJklaOaYuYkm2T/KmJBuSnJvkWUk+mOSE/v7rJHlpkrOSXJLkv5I8oL9vN+Ck/qnO79fIThjjdUiSJm/qIgYcA9wbeAhwX+DOwL3m3X98f/+fAXcE3gh8IMmdgTOBh/WP2xvYFThsmLElSUNbPfYA8yXZATgUeExVfbRf9pfAWf3tPYBHAbtV1Y/7Lzs2yR8AT6yqv07y8375eVV1wRa+z3pgPcC2bDex1yNJmqypihiwB7AG+PLcgqq6JMm3+k/vAgQ4Ncn8r1sLfGJrv0lVHQccB7BTdq5rObMkaSTTFrG5Mm0pLNv0990NuGzBfb+e1FCSpOk0bRE7nS5O+wE/BEiyHXAH4PvAKXShu0lVnbSF59jUf1w12VElSWObqh07qmoD8AbgpUkOTnJ74PX0a2BVdRrwVuCEJA9PsnuSdUkOT/LQ/ml+RLe2dkiSG/Xvs0mSZtBURax3OPAZ4P10u8t/AzgZuLS///F0eygeBXwH+CBwIF28qKqfAM8FjgTOBY4dcHZJ0oBSNd37NSRZSxeol1XVMcv9/Dtl57p7Dl7up5UkLZMv1cf5Zf08i903be+JkWRf4HZ0eyjuCBzRf3zHmHNJkqbP1EWs9wxgL+A3wNeAA6vqrHFHkiRNm6mLWFWdAqwbew5J0vSbxh07JEnaKkZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJatbqsQcYW7bZhm2uu93YYwzmOy+/w9gjDO42b9w49giD2njDtWOPMLjtP/WdsUcY1OZf/nLsEaaGa2KSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs2YuYknuk6SS3HDsWSRJkzVzEZMkrRxTF7Eka5O8Ism5SS5N8sUkB/T3XWktK8lu/bJ1SXYDTurvOr9ffsLgL0KSNIipixhwFPCnwKHAvsA3gQ8n2XUrvvZM4GH97b2BXYHDJjGkJGl8UxWxJNsDTwaOqKoTq+rbwJOAc4GnXN3XV9Vm4Of9p+dV1TlVddEi32d9kpOTnLypLl3GVyBJGtJURQzYA1gDfG5uQR+mLwC3X65vUlXHVdW6qlp3nWy7XE8rSRrYtEUs/cda5L4CLl/wOOiiJ0lagaYtYqcDm4AD5hYkWQXsD5wKnN8vnv/+2D4LnmNT/3HVhGaUJE2JqYpYVV0CvBZ4SZIHJ7ld//kuwGvoIncm8Lwkeya5P/D3C57mR3RrbYckuVGSHYZ7BZKkIU1VxHpHAO8Ejge+BtwJeGBVnV1VlwGPBHYHvg48H3j2/C+uqp8AzwWOpNsh5NjhRpckDWn12AMsVFUbgaf3fxa7//NceRNiFjzmhcALJzKgJGlqTOOamCRJW8WISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLUrNVjDzC2qqI2bx57jMHc9rUXjz3C4C6/7pqxRxjU+fusvH/W2bzX2CMMau2Hvzr2CMO6ih/RrolJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpo1kxFLcs8k30iyKcknx55HkjQZq8ceYEJeCXwdOAS4ZORZJEkTMpNrYsCtgU9U1ZlV9fOxh5EkTUaTEUuyNskrkpyb5NIkX0xyQJLdkhRwPeANSSrJ40YeV5I0IU1GDDgK+FPgUGBf4JvAh4HLgF2BXwFP72+/Y6QZJUkT1lzEkmwPPBk4oqpOrKpvA08CzgWeXFXnAAVcVFXnVNWvF3mO9UlOTnLyZXXpoPNLkpZPcxED9gDWAJ+bW1BVm4EvALffmieoquOqal1VrVuTbSczpSRp4lqMWPqPtch9iy2TJM2oFiN2OrAJOGBuQZJVwP7AqWMNJUkaXnPHiVXVJUleC7wkyQXAD4H/BewCvGbU4SRJg2ouYr0j+o/HA9cHTgEeWFVnjzeSJGloTUasqjbS7UL/9C3cv8OwE0mSxtDie2KSJAFGTJLUMCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc1aPfYAo6uiNm4ce4rB1Ne/PfYIg9tmxx3HHmFQN9lhz7FHGFz+7vyxRxjUqq/cYOwRBpULtpwq18QkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc1alogl2SbJ65L8LEklOSPJB5fjuSVJ2pLVy/Q8DwYeD9wH+AHwayDL9NySJC1quSJ2a+Dsqvr8Mj3fVklynaraNOT3lCRNj2u9OTHJCcA/A7eYtynxhPmbE5Nsn+RNSTYkOTfJs5J8sP/auceckeTwBc/9ySTHLnjM85K8IckvgLf2y2+W5O1JLuz/nJjkNtf2tUmSpttyvCd2GPAC4CxgV+BuizzmGODewEOA+wJ3Bu61xO/3DOA7wDrg2Um2A04CLu2/x/7A2cDH+vskSTPqWm9OrKqLklwMbK6qcwCSK94OS7IDcCjwmKr6aL/sL+mitxSfqqqj5j3/oXTvvz2+qqpf9kTgPOAPgXcufIIk64H1ANti5ySpVcv1nthV2QNYA3x5bkFVXZLkW0t8vpMXfH5X4FbAxfPjCWzXf+8rqarjgOMAdsrOtcQ5JEkjGyJic2W5ulhczpX3aFyzyOMuWfD5NsDXgEcu8tifX+10kqRmDXGw8+nAZcB+cwv696rusOBx59O9pzb3mG2B227F83+Vbu/IC6rq9AV/jJgkzbCJR6yqNgBvAF6a5OAktwde33/v+WtnnwD+PMl9kuzdf81ia2ILvRU4F3hfknsnuVWSA5Mc4x6KkjTbhticCHA4sD3wfmAD3S75u9DtUTjnxcBuwPv6xxwJ3PTqnriqfpXkQOAlwLuA6wE/pdtj8cJlewWSpKmTfoe+Yb9pshb4EfCyqjpm8AHm2Sk7191z8JgjaMK22XHHsUcY1Ma77zn2CIPL350/9giDWvvnl179g2bIFy54Fxdddt6iZ4EaZE0syb7A7ej2UNwROKL/+I4hvr8kaTYNtTkRuoOU9wJ+Q7c34YFVtdRjxSRJGiZiVXUK3Rk2JElaNl5PTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKatXrsAaRJu/zii8ceYVBrv3Ta2CMM7lm7f2bsEQb10tUPHnuEYSVbvMs1MUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGbNRMSSnJDkg2PPIUka1uqxB1gmhwEBSPJJ4FtV9dRRJ5IkTdxMRKyqLhp7BknS8GYiYklOAG4IXADcG7h3kqf0d9+qqs4YaTRJ0gTNRMTmOQzYE/gO8Ox+2fnjjSNJmqSZilhVXZRkE/Crqjpn7HkkSZM1UxHbWknWA+sBtmW7kaeRJC3VTOxif01V1XFVta6q1q1h7djjSJKWaBYjtglYNfYQkqTJm8WInQHsl2S3JDdMMouvUZLEbEbsaLq1sVPp9ky8xbjjSJImZSZ27Kiqx827fRqw/3jTSJKGMotrYpKkFcKISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLUrNVjDyBpeV2+YcPYIwzusR97wtgjDGrt+jVjjzCojf9ynS3e55qYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1KyZi1iS+ySpJDccexZJ0mTNXMQkSSvH1EUsydokr0hybpJLk3wxyQH9fVday0qyW79sXZLdgJP6u87vl58w+IuQJA1i6iIGHAX8KXAosC/wTeDDSXbdiq89E3hYf3tvYFfgsEkMKUka31RFLMn2wJOBI6rqxKr6NvAk4FzgKVf39VW1Gfh5/+l5VXVOVV20yPdZn+TkJCdfxsZlfAWSpCFNVcSAPYA1wOfmFvRh+gJw++X6JlV1XFWtq6p1a1i7XE8rSRrYtEUs/cda5L4CLl/wOOiiJ0lagaYtYqcDm4AD5hYkWQXsD5wKnN8vnv/+2D4LnmNT/3HVhGaUJE2JqYpYVV0CvBZ4SZIHJ7ld//kuwGvoIncm8Lwkeya5P/D3C57mR3RrbYckuVGSHYZ7BZKkIU1VxHpHAO8Ejge+BtwJeGBVnV1VlwGPBHYHvg48H3j2/C+uqp8AzwWOpNsh5NjhRpckDWn12AMsVFUbgaf3fxa7//NceRNiFjzmhcALJzKgJGlqTOOamCRJW8WISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLUrNVjDyBpmVWNPcHgbv+ic8ceYVDP/sT/HXuEQT3hXedt8T7XxCRJzTJikhzVdHIAAAcpSURBVKRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktSs1WMPMIYk64H1ANuy3cjTSJKWakWuiVXVcVW1rqrWrWHt2ONIkpZoRUZMkjQbjJgkqVkzG7EkT03ynbHnkCRNzsxGDLghsNfYQ0iSJmdmI1ZVz6uqjD2HJGlyZjZikqTZZ8QkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWrW6rEHkKRr6zc/OnPsEQZ1z21X1vrHDskW71tZ/yUkSTPFiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs4yYJKlZRkyS1CwjJklqlhGTJDWrmYglOTzJGWPPIUmaHs1ETJKkhZYlYkl2SnL95Xiua/A9b5Rk2yG/pyRpuiw5YklWJXlAkrcB5wB37pdfL8lxSc5LcnGSTyVZN+/rHpdkQ5KDk3wrySVJTkpyqwXP/7dJzukf+yZghwUjPBg4p/9e91zq65AktesaRyzJ3kmOAn4MvAO4BHgg8OkkAU4Ebgb8IbAv8GngE0l2nfc0a4FnAYcC+wPXB/5l3vd4BPBPwHOBuwDfBZ6xYJS3An8G7Ah8NMnpSf5xYQwlSbNrqyKW5AZJnpbkZOAU4LbA04FdquoJVfXpqirgIGAf4OFV9eWqOr2q/gH4AfDoeU+5GnhK/5hvAEcDByWZm+fpwBur6nVVdVpVHQl8ef5MVfWbqvpQVT0K2AV4Uf/9v9ev/R2aZOHa29zrWZ/k5CQnX8bGrflPIEmaQlu7JvY3wCuBjcBtquqPq+pdVbWwAHcFtgPO7zcDbkiyAbgDsMe8x22squ/O+/ynwBq6NTKA2wFfWPDcCz//raq6uKreUFUHAXcDbgz8G/DwLTz+uKpaV1Xr1rD2Kl62JGmard7Kxx0HXAY8BvjvJO8F3gx8vKo2z3vcNsC5wL0WeY5fzrv9mwX31byvv8aSrAUOoVvbezDw33Rrc+9byvNJktqwVdGoqp9W1ZFVtRfwB8AG4O3AWUmOSbJv/9Cv0m3au7zflDj/z3nXYK5vA/dYsOx3Pk/ngCSvo9ux5FjgdOCuVXWXqnplVV14Db6nJKkx13jNp6q+WFVPBnal28y4J/DlJPcCPgZ8DnhfkgcluVWS/ZM8v79/a70SeGySJyS5TZJnAXdf8Ji/AP4T2Al4FHDzqnpmVX3rmr4mSVKbtnZz4pX074e9G3h3khsDm6uqkjyYbs/Cf6V7b+pcurC96Ro89zuS7A4cSfce2/uBlwOPm/ewjwM3qapfXvkZJEkrQbqdCleunbJz3T0Hjz2GpGsjGXuCQX3kJ6eMPcKg9nvAmZz89UsX/Z/saackSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWrW6rEHkKRrrWrsCQb1gJvuM/YIgzqtfrbF+1wTkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWUZMktQsIyZJapYRkyQ1y4hJkpplxCRJzTJikqRmGTFJUrOMmCSpWavHHmAMSdYD6wG2ZbuRp5EkLdWKXBOrquOqal1VrVvD2rHHkSQt0YqMmCRpNhgxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLULCMmSWqWEZMkNcuISZKaZcQkSc0yYpKkZhkxSVKzjJgkqVlGTJLUrFTV2DOMKsn5wI9G+NY3BC4Y4fuOaaW9Zl/v7Ftpr3ms13vLqrrRYnes+IiNJcnJVbVu7DmGtNJes6939q201zyNr9fNiZKkZhkxSVKzjNh4jht7gBGstNfs6519K+01T93r9T0xSVKzXBOTJDXLiEmSmmXEJEnNMmKSpGYZMUlSs/4/f+93hZf+W2UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# wrong translation\n",
    "translate(u'trata de averiguarlo.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RTe5P5ioMJwN"
   },
   "source": [
    "## Next steps\n",
    "\n",
    "* [Download a different dataset](http://www.manythings.org/anki/) to experiment with translations, for example, English to German, or English to French.\n",
    "* Experiment with training on a larger dataset, or using more epochs\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "nmt_with_attention.ipynb",
   "private_outputs": true,
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python (py37-tf2)",
   "language": "python",
   "name": "py37-tf2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
